Search

In this paper, we propose an efficient derivative-free version of a martingale neural network SOC-MartNet proposed in Cai et al. [2] for solving high-dimensional Hamilton-Jacobi-Bellman (HJB) equations and stochastic optimal control problems (SOCPs) with controls on both drift and volatility. The solution of the HJB equation consists of two steps: (1) finding the optimal control from the value function, and (2) deriving the value function from a linear PDE characterized by the optimal control. The linear PDE is reformulated into a weak form of a new martingale formulation from the original SOC-MartNet where all temporal and spatial derivatives are replaced by an univariate, first-order random finite difference operator approximation, giving the derivative free version of the SOC-MartNet. Then, the optimal feedback control is identified by minimizing the mean of the value function, thereby avoiding the need for pointwise minimization on the Hamiltonian. Finally, the optimal control and value function are approximated by neural networks trained via adversarial learning using the derivative-free formulation. This method eliminates the reliance on automatic differentiation for computing temporal and spatial derivatives, offering significant efficiency in solving high-dimensional HJB equations and SOCPs.

In this paper, we derive diffusion equation models in the spectral domain for the evolution of training errors of two-layer multi-scale deep neural networks (MscaleDNN) \cite{caixu2019,liu2020multi}, designed to reduce the spectral bias of fully connected deep neural networks in approximating oscillatory functions. The diffusion models are obtained from the spectral form of the error equation of the MscaleDNN, derived with a neural tangent kernel approach and gradient descent training and a sine activation function, assuming a vanishing learning rate and infinite network width and domain size. The involved diffusion coefficients are shown to have larger supports if more scales are used in the MscaleDNN, and thus, the proposed diffusion equation models in the frequency domain explain the MscaleDNN's spectral bias reduction capability. Numerical results of the diffusion models for a two-layer MscaleDNN training match with the error evolution of actual gradient descent training with a reasonably large network width, thus validating the effectiveness of the diffusion models. Meanwhile, the numerical results for MscaleDNN show error decay over a wide frequency range and confirm the advantage of using the MscaleDNN in approximating functions with a wide range of frequencies.

In this paper, we propose forward and backward stochastic differential equations (FBSDEs) based deep neural network (DNN) learning algorithms for the solution of high dimensional quasilinear parabolic partial differential equations (PDEs), which are related to the FBSDEs by the Pardoux-Peng theory. The algorithms rely on a learning process by minimizing the pathwise difference between two discrete stochastic processes, defined by the time discretization of the FBSDEs and the DNN representation of the PDE solutions, respectively. The proposed algorithms are shown to generate DNN solutions for a 100-dimensional Black--Scholes--Barenblatt equation, accurate in a finite region in the solution space, and has a convergence rate similar to that of the Euler--Maruyama discretization used for the FBSDEs. As a result, a Richardson extrapolation technique over time discretizations can be used to enhance the accuracy of the DNN solutions. For time oscillatory solutions, a multiscale DNN is shown to improve the performance of the FBSDE DNN for high frequencies.

In this paper, we study a multi-scale deep neural network (MscaleDNN) as a meshless numerical method for computing oscillatory Stokes flows in complex domains. The MscaleDNN employs a multi-scale structure in the design of its DNN using radial scalings to convert the approximation of high frequency components of the highly oscillatory Stokes solution to one of lower frequencies. The MscaleDNN solution to the Stokes problem is obtained by minimizing a loss function in terms of L2 normof the residual of the Stokes equation. Three forms of loss functions are investigated based on vorticity-velocity-pressure, velocity-stress-pressure, and velocity-gradient of velocity-pressure formulations of the Stokes equation. We first conduct a systematic study of the MscaleDNN methods with various loss functions on the Kovasznay flow in comparison with normal fully connected DNNs. Then, Stokes flows with highly oscillatory solutions in a 2-D domain with six randomly placed holes are simulated by the MscaleDNN. The results show that MscaleDNN has faster convergence and consistent error decays in the simulation of Kovasznay flow for all four tested loss functions. More importantly, the MscaleDNN is capable of learning highly oscillatory solutions when the normal DNNs fail to converge.

A matrix basis formulation is introduced to represent the 3 x 3 dyadic Green's functions in the frequency domain for the Maxwell's equations and the elastic wave equation in layered media. The formulation can be used to decompose the Maxwell's Green's functions into independent TE and TM components, each satisfying a Helmholtz equation, and decompose the elastic wave Green's function into the S-wave and the P-wave components. In addition, a derived vector basis formulation is applied to the case for acoustic wave sources from a non-viscous fluid layer.

In this paper, we prove the exponential convergence of the multipole and local expansions, shifting and translation operators used in fast multipole methods (FMMs) for 3-dimensional Laplace equations in layered media. These theoretical results ensure the exponential convergence of the FMM which has been shown by the numerical results recently reported in [9]. As the free space components are calculated by the classic FMM, this paper will focus on the analysis for the reaction components of the Green's function for the Laplace equation in layered media. We first prove that the density functions in the integral representations of the reaction components are analytic and bounded in the right half complex plane. Then, using the Cagniard-de Hoop transform and contour deformations, estimate for the remainder terms of the truncated expansions is given, and, as a result, the exponential convergence for the expansions and translation operators is proven.

In this paper, we develop fast multipole methods for 3D Helmholtz kernel in layered media. Two algorithms based on different forms of Taylor expansion of layered media Green's function are developed. A key component of the first algorithm is an efficient algorithm based on discrete complex image approximation and recurrence formula for the calculation of the layered media Green's function and its derivatives, which are given in terms of Sommerfeld integrals. The second algorithm uses symmetric derivatives in the Taylor expansion to reduce the size of precomputed tables for the derivatives of layered media Green's function. Numerical tests in layered media have validated the accuracy and O(N) complexity of the proposed algorithms.

In this paper, a fast multipole method (FMM) is proposed to compute long-range interactions of wave sources embedded in 3-D layered media. The layered media Green's function for the Helmholtz equation, which satisfies the transmission conditions at material interfaces, is decomposed into a free space component and four types of reaction field components arising from wave reflections and transmissions through the layered media. The proposed algorithm is a combination of the classic FMM for the free space component and FMMs specifically designed for the four types reaction components, made possible by new multipole expansions (MEs) and local expansions (LEs) as well as the multipole-to-local translation (M2L) operators for the reaction field components. { Moreover, equivalent polarization source can be defined for each reaction component based on the convergence analysis of its ME. The FMMs for the reaction components, implemented with the target particles and equivalent polarization sources, are found to be much more efficient than the classic FMM for the free space component due to the fact that the equivalent polarization sources and the target particles are always separated by a material interface.} As a result, the FMM algorithm developed for layered media has a similar computational cost as that for the free space. Numerical results validate the fast convergence of the MEs and the $O(N)$ complexity of the FMM for interactions of low-frequency wave sources in 3-D layered media.

In this paper, we will first give a derivation of the multipole expansion (ME) and local expansion (LE) for the far field from sources in general 2-D layered media and the multipole-to-local translation (M2L) operator by using the generating function for Bessel functions. Then, we present a rigorous proof of the exponential convergence of the ME, LE, and M2L for 2-D Helmholtz equations in layered media. It is shown that the convergence of ME, LE, and M2L for the reaction field component of the Green's function depends on a polarized distance between the target and a polarized image of the source.