Search

We introduce a conceptual framework for numerically solving linear elliptic, parabolic, and hyperbolic PDEs on bounded, polytopal domains in euclidean spaces by deep neural networks. The PDEs are recast as minimization of a least-squares (LSQ for short) residual of an equivalent, well-posed first-order system, over parametric families of deep neural networks. The associated LSQ residual is a) equal or proportional to a weak residual of the PDE, b) additive in terms of contributions from localized subnetworks, indicating locally ``out-of-equilibrium'' of neural networks with respect to the PDE residual, c) serves as numerical loss function for neural network training, and d) constitutes, even with incomplete training, a computable, (quasi-)optimal numerical error estimator in the context of adaptive LSQ finite element methods. In addition, an adaptive neural network growth strategy is proposed which, assuming exact numerical minimization of the LSQ loss functional, yields sequences of neural networks with realizations that converge rate-optimally to the exact solution of the first order system LSQ formulation.

We study the approximation rates of a class of deep neural network approximations of operators, which arise as data-to-solution maps $\mathcal{G}^\dagger$ of linear elliptic partial differential equations (PDEs), and act between pairs $X,Y$ of suitable infinite-dimensional spaces. We prove expression rate bounds for approximate neural operators $\mathcal{G}$ with the structure $\mathcal{G} = \mathcal{R} \circ \mathcal{A} \circ \mathcal{E}$, with linear encoders $\mathcal{E}$ and decoders $\mathcal{R}$. The constructive proofs are via a recurrent NN structure obtained by unrolling exponentially convergent, self-consistent (``Richardson'') iterations. We bound the operator approximation error with respect to the linear Kolmogorov $N$-widths of the data and solution sets and in terms of the size of the approximation network. We prove expression rate bounds for approximate, neural solution operators emulating the coefficient-to-solution maps for elliptic PDEs set in $d$-dimensional polytopes, with $d\in\{2,3\}$, and subject to Dirichlet-, Neumann- or mixed boundary conditions. Exploiting weighted norm characterizations of the solution sets of elliptic PDEs in polytopes, we show algebraic rates of expression for problems with data with finite regularity, and exponential operator expression rates for analytic data.

We consider frequency-domain acoustic scattering at a homogeneous star-shaped penetrable obstacle, whose shape is uncertain and modelled via a radial spectral parameterization with random coefficients. Using recent results on the stability of Helmholtz transmission problems with piecewise constant coefficients from [A. Moiola and E. A. Spence, Acoustic transmission problems: wavenumber-explicit bounds and resonance-free regions, Mathematical Models and Methods in Applied Sciences, 29 (2019), pp. 317-354] we obtain frequency-explicit statements on the holomorphic dependence of the scattered field and the far-field pattern on the stochastic shape parameters. This paves the way for applying general results on the efficient construction of high-dimensional surrogate models. We also take into account the effect of domain truncation by means of perfectly matched layers (PML). In addition, spatial regularity estimates which are explicit in terms of the wavenumber $k$ permit us to quantify the impact of finite-element Galerkin discretization using high-order Lagrangian finite-element spaces.

We obtain wavenumber-robust error bounds for the deep neural network (DNN) emulation of the solution to the time-harmonic, sound-soft acoustic scattering problem in the exterior of a smooth, convex obstacle in two physical dimensions. The error bounds are based on a boundary reduction of the scattering problem in the unbounded exterior region to its smooth, curved boundary $\Gamma$ using the so-called combined field integral equation (CFIE), a well-posed, second-kind boundary integral equation (BIE) for the field's Neumann datum on $\Gamma$. In this setting, the continuity and stability constants of this formulation are explicit in terms of the (non-dimensional) wavenumber $\kappa$. Using wavenumber-explicit asymptotics of the problem's Neumann datum, we analyze the DNN approximation rate for this problem. We use fully connected NNs of the feed-forward type with Rectified Linear Unit (ReLU) activation. Through a constructive argument we prove the existence of DNNs with an $\epsilon$-error bound in the $L^\infty(\Gamma)$-norm having a small, fixed width and a depth that increases $\textit{spectrally}$ with the target accuracy $\epsilon>0$. We show that for fixed $\epsilon>0$, the depth of these NNs should increase $\textit{poly-logarithmically}$ with respect to the wavenumber $\kappa$ whereas the width of the NN remains fixed. Unlike current computational approaches, such as wavenumber-adapted versions of the Galerkin Boundary Element Method (BEM) with shape- and wavenumber-tailored solution $\textit{ansatz}$ spaces, our DNN approximations do not require any prior analytic information about the scatterer's shape.

We study the fully explicit numerical approximation of a semilinear elliptic boundary value model problem, which features a monomial reaction and analytic forcing, in a bounded polygon $\Omega\subset\mathbb{R}^2$ with a finite number of straight edges. In particular, we analyze the convergence of $hp$-type iterative linearized Galerkin ($hp$-ILG) solvers. Our convergence analysis is carried out for conforming $hp$-finite element (FE) Galerkin discretizations on sequences of regular, simplicial partitions of $\Omega$, with geometric corner refinement, with polynomial degrees increasing in sync with the geometric mesh refinement towards the corners of $\Omega$. For a sequence of discrete solutions generated by the ILG solver, with a stopping criterion that is consistent with the exponential convergence of the exact $hp$-FE Galerkin solution, we prove exponential convergence in $\mathrm{H}^1(\Omega)$ to the unique weak solution of the boundary value problem. Numerical experiments illustrate the exponential convergence of the numerical approximations obtained from the proposed scheme in terms of the number of degrees of freedom as well as of the computational complexity involved.

We analyze deep Neural Network emulation rates of smooth functions with point singularities in bounded, polytopal domains $\mathrm{D} \subset \mathbb{R}^d$, $d=2,3$. We prove exponential emulation rates in Sobolev spaces in terms of the number of neurons and in terms of the number of nonzero coefficients for Gevrey-regular solution classes defined in terms of weighted Sobolev scales in $\mathrm{D}$, comprising the countably-normed spaces of I.M. Babu\v{s}ka and B.Q. Guo. As intermediate result, we prove that continuous, piecewise polynomial high order (``$p$-version'') finite elements with elementwise polynomial degree $p\in\mathbb{N}$ on arbitrary, regular, simplicial partitions of polyhedral domains $\mathrm{D} \subset \mathbb{R}^d$, $d\geq 2$ can be exactly emulated by neural networks combining ReLU and ReLU$^2$ activations. On shape-regular, simplicial partitions of polytopal domains $\mathrm{D}$, both the number of neurons and the number of nonzero parameters are proportional to the number of degrees of freedom of the finite element space, in particular for the $hp$-Finite Element Method of I.M. Babu\v{s}ka and B.Q. Guo.

On a finite time interval $(0,T)$, we consider the multiresolution Galerkin discretization of a modified Hilbert transform $\mathcal H_T$ which arises in the space-time Galerkin discretization of the linear diffusion equation. To this end, we design spline-wavelet systems in $(0,T)$ consisting of piecewise polynomials of degree $\geq 1$ with sufficiently many vanishing moments which constitute Riesz bases in the Sobolev spaces $ H^{s}_{0,}(0,T)$ and $ H^{s}_{,0}(0,T)$. These bases provide multilevel splittings of the temporal discretization spaces into "increment" or "detail" spaces of direct sum type. Via algebraic tensor-products of these temporal multilevel discretizations with standard, hierarchic finite element spaces in the spatial domain (with standard Lagrangian FE bases), sparse space-time tensor-product spaces are obtained, which afford a substantial reduction in the number of the degrees of freedom as compared to time-marching discretizations. In addition, temporal spline-wavelet bases allow to compress certain nonlocal integrodifferential operators which appear in stable space-time variational formulations of initial-boundary value problems, such as the heat equation and the acoustic wave equation. An efficient preconditioner is proposed that affords linear complexity solves of the linear system of equations which results from the sparse space-time Galerkin discretization., Comment: 32 pages

We prove deep neural network (DNN for short) expressivity rate bounds for solution sets of a model class of singularly perturbed, elliptic two-point boundary value problems, in Sobolev norms, on the bounded interval $(-1,1)$. We assume that the given source term and reaction coefficient are analytic in $[-1,1]$. We establish expression rate bounds in Sobolev norms in terms of the NN size which are uniform with respect to the singular perturbation parameter for several classes of DNN architectures. In particular, ReLU NNs, spiking NNs, and $\tanh$- and sigmoid-activated NNs. The latter activations can represent ``exponential boundary layer solution features'' explicitly, in the last hidden layer of the DNN, i.e. in a shallow subnetwork, and afford improved robust expression rate bounds in terms of the NN size. We prove that all DNN architectures allow robust exponential solution expression in so-called `energy' as well as in `balanced' Sobolev norms, for analytic input data.

We show expression rates and stability in Sobolev norms of deep feedforward ReLU neural networks (NNs) in terms of the number of parameters defining the NN for continuous, piecewise polynomial functions, on arbitrary, finite partitions $\mathcal{T}$ of a bounded interval $(a,b)$. Novel constructions of ReLU NN surrogates encoding function approximations in terms of Chebyshev polynomial expansion coefficients are developed which require fewer neurons than previous constructions. Chebyshev coefficients can be computed easily from the values of the function in the Clenshaw--Curtis points using the inverse fast Fourier transform. Bounds on expression rates and stability are obtained that are superior to those of constructions based on ReLU NN emulations of monomials as considered in [Opschoor, Petersen and Schwab, 2020] and [Montanelli, Yang and Du, 2021]. All emulation bounds are explicit in terms of the (arbitrary) partition of the interval, the target emulation accuracy and the polynomial degree in each element of the partition. ReLU NN emulation error estimates are provided for various classes of functions and norms, commonly encountered in numerical analysis. In particular, we show exponential ReLU emulation rate bounds for analytic functions with point singularities and develop an interface between Chebfun approximations and constructive ReLU NN emulations.

This article is concerned with a regularity analysis of parametric operator equations with a perspective on uncertainty quantification. We study the regularity of mappings between Banach spaces near branches of isolated solutions that are implicitly defined by a residual equation. Under $s$-Gevrey assumptions on on the residual equation, we establish $s$-Gevrey bounds on the Fr\'echet derivatives of the local data-to-solution mapping. This abstract framework is illustrated in a proof of regularity bounds for a semilinear elliptic partial differential equation with parametric and random field input.

On polytopal domains in $\mathbb{R}^3$, we prove weighted analytic regularity of solutions to the Dirichlet problem for the integral fractional Laplacian with analytic right-hand side. Employing the Caffarelli-Silvestre extension allows to localize the problem and to decompose the regularity estimates into results on vertex, edge, face, vertex-edge, vertex-face, edge-face and vertex-edge-face neighborhoods of the boundary. Using tangential differentiability of the extended solutions, a bootstrapping argument based on Caccioppoli inequalities on dyadic decompositions of the neighborhoods provides weighted, analytic control of higher order solution derivatives., Comment: arXiv admin note: text overlap with arXiv:2112.08151

We establish universality and expression rate bounds for a class of neural Deep Operator Networks (DON) emulating Lipschitz (or H\"older) continuous maps $\mathcal G:\mathcal X\to\mathcal Y$ between (subsets of) separable Hilbert spaces $\mathcal X$, $\mathcal Y$. The DON architecture considered uses linear encoders $\mathcal E$ and decoders $\mathcal D$ via (biorthogonal) Riesz bases of $\mathcal X$, $\mathcal Y$, and an approximator network of an infinite-dimensional, parametric coordinate map that is Lipschitz continuous on the sequence space $\ell^2(\mathbb N)$. Unlike previous works ([Herrmann, Schwab and Zech: Neural and Spectral operator surrogates: construction and expression rate bounds, SAM Report, 2022], [Marcati and Schwab: Exponential Convergence of Deep Operator Networks for Elliptic Partial Differential Equations, SAM Report, 2022]), which required for example $\mathcal G$ to be holomorphic, the present expression rate results require mere Lipschitz (or H\"older) continuity of $\mathcal G$. Key in the proof of the present expression rate bounds is the use of either super-expressive activations (e.g. [Yarotski: Elementary superexpressive activations, Int. Conf. on ML, 2021], [Shen, Yang and Zhang: Neural network approximation: Three hidden layers are enough, Neural Networks, 2021], and the references there) which are inspired by the Kolmogorov superposition theorem, or of nonstandard NN architectures with standard (ReLU) activations as recently proposed in [Zhang, Shen and Yang: Neural Network Architecture Beyond Width and Depth, Adv. in Neural Inf. Proc. Sys., 2022]. We illustrate the abstract results by approximation rate bounds for emulation of a) solution operators for parametric elliptic variational inequalities, and b) Lipschitz maps of Hilbert-Schmidt operators., Comment: 31 pages

We develop a multilevel Monte Carlo (MLMC)-FEM algorithm for linear, elliptic diffusion problems in polytopal domain $\mathcal D\subset \mathbb R^d$, with Besov-tree random coefficients. This is to say that the logarithms of the diffusion coefficients are sampled from so-called Besov-tree priors, which have recently been proposed to model data for fractal phenomena in science and engineering. Numerical analysis of the fully discrete FEM for the elliptic PDE includes quadrature approximation and must account for a) nonuniform pathwise upper and lower coefficient bounds, and for b) low path-regularity of the Besov-tree coefficients. Admissible non-parametric random coefficients correspond to random functions exhibiting singularities on random fractals with tunable fractal dimension, but involve no a-priori specification of the fractal geometry of singular supports of sample paths. Optimal complexity and convergence rate estimates for quantities of interest and for their second moments are proved. A convergence analysis for MLMC-FEM is performed which yields choices of the algorithmic steering parameters for efficient implementation. A complexity (``error vs work'') analysis of the MLMC-FEM approximations is provided., Comment: 41 pages

We propose a novel a-posteriori error estimation technique where the target quantities of interest are ratios of high-dimensional integrals, as occur e.g. in PDE constrained Bayesian inversion and PDE constrained optimal control subject to an entropic risk measure. We consider in particular parametric, elliptic PDEs with affine-parametric diffusion coefficient, on high-dimensional parameter spaces. We combine our recent a-posteriori Quasi-Monte Carlo (QMC) error analysis, with Finite Element a-posteriori error estimation. The proposed approach yields a computable a-posteriori estimator which is reliable, up to higher order terms. The estimator's reliability is uniform with respect to the PDE discretization, and robust with respect to the parametric dimension of the uncertain PDE input.

We continue our study [Domain Uncertainty Quantification in Computational Electromagnetics, JUQ (2020), 8:301--341] of the numerical approximation of time-harmonic electromagnetic fields for the Maxwell lossy cavity problem for uncertain geometries. We adopt the same affine-parametric shape parametrization framework, mapping the physical domains to a nominal polygonal domain with piecewise smooth maps. The regularity of the pullback solutions on the nominal domain is characterized in piecewise Sobolev spaces. We prove error convergence rates and optimize the algorithmic steering of parameters for edge-element discretizations in the nominal domain combined with: (a) multilevel Monte Carlo sampling, and (b) multilevel, sparse-grid quadrature for computing the expectation of the solutions with respect to uncertain domain ensembles. In addition, we analyze sparse-grid interpolation to compute surrogates of the domain-to-solution mappings. All calculations are performed on the polyhedral nominal domain, which enables the use of standard simplicial finite element meshes. We provide a rigorous fully discrete error analysis and show, in all cases, that dimension-independent algebraic convergence is achieved. For the multilevel sparse-grid quadrature methods, we prove higher order convergence rates which are free from the so-called curse of dimensionality, i.e. independent of the number of parameters used to parametrize the admissible shapes. Numerical experiments confirm our theoretical results and verify the superiority of the sparse-grid methods.

We formulate standard and multilevel Monte Carlo methods for the $k$th moment $\mathbb{M}^k_\varepsilon[\xi]$ of a Banach space valued random variable $\xi\colon\Omega\to E$, interpreted as an element of the $k$-fold injective tensor product space $\otimes^k_\varepsilon E$. For the standard Monte Carlo estimator of $\mathbb{M}^k_\varepsilon[\xi]$, we prove the $k$-independent convergence rate $1-\frac{1}{p}$ in the $L_q(\Omega;\otimes^k_\varepsilon E)$-norm, provided that (i) $\xi\in L_{kq}(\Omega;E)$ and (ii) $q\in[p,\infty)$, where $p\in[1,2]$ is the Rademacher type of $E$. By using the fact that Rademacher averages are dominated by Gaussian sums combined with a version of Slepian's inequality for Gaussian processes due to Fernique, we moreover derive corresponding results for multilevel Monte Carlo methods, including a rigorous error estimate in the $L_q(\Omega;\otimes^k_\varepsilon E)$-norm and the optimization of the computational cost for a given accuracy. Whenever the type of the Banach space $E$ is $p=2$, our findings coincide with known results for Hilbert space valued random variables. We illustrate the abstract results by three model problems: second-order elliptic PDEs with random forcing or random coefficient, and stochastic evolution equations. In these cases, the solution processes naturally take values in non-Hilbertian Banach spaces. Further applications, where physical modeling constraints impose a setting in Banach spaces of type $p<2$, are indicated., Comment: 47 pages

We prove exponential convergence in the energy norm of $hp$ finite element discretizations for the integral fractional diffusion operator of order $2s\in (0,2)$ subject to homogeneous Dirichlet boundary conditions in bounded polygonal domains $\Omega\subset \mathbb{R}^2$. Key ingredient in the analysis are the weighted analytic regularity from our previous work and meshes that feature anisotropic geometric refinement towards $\partial\Omega$.

Approximation rates are analyzed for deep surrogates of maps between infinite-dimensional function spaces, arising e.g. as data-to-solution maps of linear and nonlinear partial differential equations. Specifically, we study approximation rates for Deep Neural Operator and Generalized Polynomial Chaos (gpc) Operator surrogates for nonlinear, holomorphic maps between infinite-dimensional, separable Hilbert spaces. Operator in- and outputs from function spaces are assumed to be parametrized by stable, affine representation systems. Admissible representation systems comprise orthonormal bases, Riesz bases or suitable tight frames of the spaces under consideration. Algebraic expression rate bounds are established for both, deep neural and spectral operator surrogates acting in scales of separable Hilbert spaces containing domain and range of the map to be expressed, with finite Sobolev or Besov regularity. We illustrate the abstract concepts by expression rate bounds for the coefficient-to-solution map for a linear elliptic PDE on the torus.

We prove weighted analytic regularity for the solution of the integral fractional Poisson problem on bounded intervals with analytic right-hand side. Based on this regularity result, we prove exponential convergence of the hp-FEM on geometric boundary-refined meshes.

For linear parabolic initial-boundary value problems with self-adjoint, time-homogeneous elliptic spatial operator in divergence form with Lipschitz-continuous coefficients, and for incompatible, time-analytic forcing term in polygonal/polyhedral domains $D$, we prove time-analyticity of solutions. Temporal analyticity is quantified in terms of weighted, analytic function classes, for data with finite, low spatial regularity and without boundary compatibility. Leveraging this result, we prove exponential convergence of a conforming, semi-discrete $hp$-time-stepping approach. We combine this semi-discretization in time with first-order, so-called "$h$-version" Lagrangian Finite Elements with corner-refinements in space into a tensor-product, conforming discretization of a space-time formulation. We prove that, under appropriate corner- and corner-edge mesh-refinement of $D$, error vs. number of degrees of freedom in space-time behaves essentially (up to logarithmic terms), to what standard FEM provide for one elliptic boundary value problem solve in $D$. We focus on two-dimensional spatial domains and comment on the one- and the three-dimensional case., Comment: 42 pages

On general regular simplicial partitions $\mathcal{T}$ of bounded polytopal domains $\Omega \subset \mathbb{R}^d$, $d\in\{2,3\}$, we construct \emph{exact neural network (NN) emulations} of all lowest order finite element spaces in the discrete de Rham complex. These include the spaces of piecewise constant functions, continuous piecewise linear (CPwL) functions, the classical ``Raviart-Thomas element'', and the ``N\'{e}d\'{e}lec edge element''. For all but the CPwL case, our network architectures employ both ReLU (rectified linear unit) and BiSU (binary step unit) activations to capture discontinuities. In the important case of CPwL functions, we prove that it suffices to work with pure ReLU nets. Our construction and DNN architecture generalizes previous results in that no geometric restrictions on the regular simplicial partitions $\mathcal{T}$ of $\Omega$ are required for DNN emulation. In addition, for CPwL functions our DNN construction is valid in any dimension $d\geq 2$. Our ``FE-Nets'' are required in the variationally correct, structure-preserving approximation of boundary value problems of electromagnetism in nonconvex polyhedra $\Omega \subset \mathbb{R}^3$. They are thus an essential ingredient in the application of e.g., the methodology of ``physics-informed NNs'' or ``deep Ritz methods'' to electromagnetic field simulation via deep learning techniques. We indicate generalizations of our constructions to higher-order compatible spaces and other, non-compatible classes of discretizations, in particular the ``Crouzeix-Raviart'' elements and Hybridized, Higher Order (HHO) methods.

We establish sparsity and summability results for coefficient sequences of Wiener-Hermite polynomial chaos expansions of countably-parametric solutions of linear elliptic and parabolic divergence-form partial differential equations with Gaussian random field inputs. The novel proof technique developed here is based on analytic continuation of parametric solutions into the complex domain. It differs from previous works that used bootstrap arguments and induction on the differentiation order of solution derivatives with respect to the parameters. The present holomorphy-based argument allows a unified, ``differentiation-free'' proof of sparsity (expressed in terms of $\ell^p$-summability or weighted $\ell^2$-summability) of sequences of Wiener-Hermite coefficients in polynomial chaos expansions in various scales of function spaces. The analysis also implies corresponding analyticity and sparsity results for posterior densities in Bayesian inverse problems subject to Gaussian priors on uncertain inputs from function spaces. Our results furthermore yield dimension-independent convergence rates of various \emph{constructive} high-dimensional deterministic numerical approximation schemes such as single-level and multi-level versions of Hermite-Smolyak anisotropic sparse-grid interpolation and quadrature in both forward and inverse computational uncertainty quantification., Comment: 165 pages

We prove weighted analytic regularity of solutions to the Dirichlet problem for the integral fractional Laplacian in polygons with analytic right-hand side. We localize the problem through the Caffarelli-Silvestre extension and study the tangential differentiability of the extended solutions, followed by bootstrapping based on Caccioppoli inequalities on dyadic decompositions of vertex, edge, and edge-vertex neighborhoods.

We construct and analyze approximation rates of deep operator networks (ONets) between infinite-dimensional spaces that emulate with an exponential rate of convergence the coefficient-to-solution map of elliptic second-order partial differential equations. In particular, we consider problems set in $d$-dimensional periodic domains, $d=1, 2, \dots$, and with analytic right-hand sides and coefficients. Our analysis covers linear, elliptic second order divergence-form PDEs as, e.g., diffusion-reaction problems, parametric diffusion equations, and elliptic systems such as linear isotropic elastostatics in heterogeneous materials. We leverage the exponential convergence of spectral collocation methods for boundary value problems whose solutions are analytic. In the present periodic and analytic setting, this follows from classical elliptic regularity. Within the ONet branch and trunk construction of [Chen and Chen, 1993] and of [Lu et al., 2021], we show the existence of deep ONets which emulate the coefficient-to-solution map to a desired accuracy in the $H^1$ norm, uniformly over the coefficient set. We prove that the neural networks in the ONet have size $\mathcal{O}(\left|\log(\varepsilon)\right|^\kappa)$, where $\varepsilon>0$ is the approximation accuracy, for some $\kappa>0$ depending on the physical space dimension.

For artificial deep neural networks, we prove expression rates for analytic functions $f:\mathbb{R}^d\to\mathbb{R}$ in the norm of $L^2(\mathbb{R}^d,\gamma_d)$ where $d\in {\mathbb{N}}\cup\{ \infty \}$. Here $\gamma_d$ denotes the Gaussian product probability measure on $\mathbb{R}^d$. We consider in particular ReLU and ReLU${}^k$ activations for integer $k\geq 2$. For $d\in\mathbb{N}$, we show exponential convergence rates in $L^2(\mathbb{R}^d,\gamma_d)$. In case $d=\infty$, under suitable smoothness and sparsity assumptions on $f:\mathbb{R}^{\mathbb{N}}\to\mathbb{R}$, with $\gamma_\infty$ denoting an infinite (Gaussian) product measure on $\mathbb{R}^{\mathbb{N}}$, we prove dimension-independent expression rate bounds in the norm of $L^2(\mathbb{R}^{\mathbb{N}},\gamma_\infty)$. The rates only depend on quantified holomorphy of (an analytic continuation of) the map $f$ to a product of strips in $\mathbb{C}^d$. As an application, we prove expression rate bounds of deep ReLU-NNs for response surfaces of elliptic PDEs with log-Gaussian random field inputs.

Centered Gaussian random fields (GRFs) indexed by compacta such as smooth, bounded Euclidean domains or smooth, compact and orientable manifolds are determined by their covariance operators. We consider centered GRFs given as variational solutions to coloring operator equations driven by spatial white noise, with an elliptic self-adjoint pseudodifferential coloring operator from the H\"ormander class. This includes the Mat\'ern class of GRFs as a special case. Using biorthogonal multiresolution analyses on the manifold, we prove that the precision and covariance operators, respectively, may be identified with bi-infinite matrices and finite sections may be diagonally preconditioned rendering the condition number independent of the dimension $p$ of this section. We prove that a tapering strategy by thresholding applied on finite sections of the bi-infinite precision and covariance matrices results in optimally numerically sparse approximations. That is, asymptotically only linearly many nonzero matrix entries are sufficient to approximate the original section of the bi-infinite covariance or precision matrix using this tapering strategy to arbitrary precision. The locations of these nonzero matrix entries are known a priori. The tapered covariance or precision matrices may also be optimally diagonally preconditioned. Analysis of the relative size of the entries of the tapered covariance matrices motivates novel, multilevel Monte Carlo (MLMC) oracles for covariance estimation, in sample complexity that scales log-linearly with respect to the number $p$ of parameters. In addition, we propose and analyze a novel compressive algorithm for simulating and kriging of GRFs. The complexity (work and memory vs. accuracy) of these three algorithms scales near-optimally in terms of the number of parameters $p$ of the sample-wise approximation of the GRF in Sobolev scales., Comment: 48 pages, 9 figures

Deep neural networks (DNNs) with ReLU activation function are proved to be able to express viscosity solutions of linear partial integrodifferental equations (PIDEs) on state spaces of possibly high dimension $d$. Admissible PIDEs comprise Kolmogorov equations for high-dimensional diffusion, advection, and for pure jump L\'{e}vy processes. We prove for such PIDEs arising from a class of jump-diffusions on $\mathbb{R}^d$ that for any suitable measure $\mu^d$ on $\mathbb{R}^d$ there exist constants $C,{\mathfrak{p}},{\mathfrak{q}}>0$ such that for every $\varepsilon \in (0,1]$ and for every $d\in \mathbb{N}$ the DNN $L^2(\mu^d)$-expression error of viscosity solutions of the PIDE is of size $\varepsilon$ with DNN size bounded by $Cd^{\mathfrak{p}}\varepsilon^{-\mathfrak{q}}$. In particular, the constant $C>0$ is independent of $d\in \mathbb{N}$ and of $\varepsilon \in (0,1]$ and depends only on the coefficients in the PIDE and the measure used to quantify the error. This establishes that ReLU DNNs can break the curse of dimensionality (CoD for short) for viscosity solutions of linear, possibly degenerate PIDEs corresponding to suitable Markovian jump-diffusion processes. As a consequence of the employed techniques we also obtain that expectations of a large class of path-dependent functionals of the underlying jump-diffusion processes can be expressed without the CoD.

We study the expression rates of deep neural networks (DNNs for short) for option prices written on baskets of $d$ risky assets, whose log-returns are modelled by a multivariate L\'evy process with general correlation structure of jumps. We establish sufficient conditions on the characteristic triplet of the L\'evy process $X$ that ensure $\varepsilon$ error of DNN expressed option prices with DNNs of size that grows polynomially with respect to $\mathcal{O}(\varepsilon^{-1})$, and with constants implied in $\mathcal{O}(\cdot)$ which grow polynomially with respect $d$, thereby overcoming the curse of dimensionality and justifying the use of DNNs in financial modelling of large baskets in markets with jumps. In addition, we exploit parabolic smoothing of Kolmogorov partial integrodifferential equations for certain multivariate L\'evy processes to present alternative architectures of ReLU DNNs that provide $\varepsilon$ expression error in DNN size $\mathcal{O}(|\log(\varepsilon)|^a)$ with exponent $a \sim d$, however, with constants implied in $\mathcal{O}(\cdot)$ growing exponentially with respect to $d$. Under stronger, dimension-uniform non-degeneracy conditions on the L\'evy symbol, we obtain algebraic expression rates of option prices in exponential L\'evy models which are free from the curse of dimensionality. In this case the ReLU DNN expression rates of prices depend on certain sparsity conditions on the characteristic L\'evy triplet. We indicate several consequences and possible extensions of the present results.

For the spectral fractional diffusion operator of order $2s\in (0,2)$ in bounded, curvilinear polygonal domains $\Omega$ we prove exponential convergence of two classes of $hp$ discretizations under the assumption of analytic data, without any boundary compatibility, in the natural fractional Sobolev norm $\mathbb{H}^s(\Omega)$. The first $hp$ discretization is based on writing the solution as a co-normal derivative of a $2+1$-dimensional local, linear elliptic boundary value problem, to which an $hp$-FE discretization is applied. A diagonalization in the extended variable reduces the numerical approximation of the inverse of the spectral fractional diffusion operator to the numerical approximation of a system of local, decoupled, second order reaction-diffusion equations in $\Omega$. Leveraging results on robust exponential convergence of $hp$-FEM for second order, linear reaction diffusion boundary value problems in $\Omega$, exponential convergence rates for solutions $u\in \mathbb{H}^s(\Omega)$ of $\mathcal{L}^s u = f$ follow. Key ingredient in this $hp$-FEM are boundary fitted meshes with geometric mesh refinement towards $\partial\Omega$. The second discretization is based on exponentially convergent sinc quadrature approximations of the Balakrishnan integral representation of $\mathcal{L}^{-s}$, combined with $hp$-FE discretizations of a decoupled system of local, linear, singularly perturbed reaction-diffusion equations in $\Omega$. The present analysis for either approach extends to polygonal subsets $\widetilde{\mathcal{M}}$ of analytic, compact $2$-manifolds $\mathcal{M}$. Numerical experiments for model problems in nonconvex polygonal domains and with incompatible data confirm the theoretical results. Exponentially small bounds on Kolmogoroff $n$-widths of solutions sets for spectral fractional diffusion in polygons are deduced.

We prove exponential expressivity with stable ReLU Neural Networks (ReLU NNs) in $H^1(\Omega)$ for weighted analytic function classes in certain polytopal domains $\Omega$, in space dimension $d=2,3$. Functions in these classes are locally analytic on open subdomains $D\subset \Omega$, but may exhibit isolated point singularities in the interior of $\Omega$ or corner and edge singularities at the boundary $\partial \Omega$. The exponential expression rate bounds proved here imply uniform exponential expressivity by ReLU NNs of solution families for several elliptic boundary and eigenvalue problems with analytic data. The exponential approximation rates are shown to hold in space dimension $d = 2$ on Lipschitz polygons with straight sides, and in space dimension $d=3$ on Fichera-type polyhedral domains with plane faces. The constructive proofs indicate in particular that NN depth and size increase poly-logarithmically with respect to the target NN approximation accuracy $\varepsilon>0$ in $H^1(\Omega)$. The results cover in particular solution sets of linear, second order elliptic PDEs with analytic data and certain nonlinear elliptic eigenvalue problems with analytic nonlinearities and singular, weighted analytic potentials as arise in electron structure models. In the latter case, the functions correspond to electron densities that exhibit isolated point singularities at the positions of the nuclei. Our findings provide in particular mathematical foundation of recently reported, successful uses of deep neural networks in variational electron structure algorithms., Comment: Found Comput Math (2022)

In a plane polygon $P$ with straight sides, we prove analytic regularity of the Leray-Hopf solution of the stationary, viscous, and incompressible Navier-Stokes equations. We assume small data, analytic volume force and no-slip boundary conditions. Analytic regularity is quantified in so-called countably normed, corner-weighted spaces with homogeneous norms. Implications of this analytic regularity include exponential smallness of Kolmogorov $N$-widths of solutions, exponential convergence rates of mixed $hp$-discontinuous Galerkin finite element and spectral element discretizations and of model order reduction techniques.