Engineering software 2.0 by Interpolating Neural Networks: Unifying Training, Solving, and Calibration

The evolution of artificial intelligence (AI) and neural network theories has revolutionized the way software is programmed, shifting from a hard-coded series of codes, Software 1.0, to a vast neural network, Software 2.0. However, this transition in engineering software has faced challenges such as data scarcity, multi-modality of data, low model accuracy, and slow inference. Here, we propose a new network based on interpolation theories and tensor decomposition, the interpolating neural network (INN) to open the new era of Engineering Software 2.0 that unifies training, solving, and calibration. Instead of interpolating training data, a common notion in computer science, INN interpolates grid points in the physical space whose coordinates and values are trainable. INN features orders of magnitude fewer trainable parameters (or degrees of freedom for solving), faster training/solving, less inference cost, smaller memory footprint, and higher model accuracy compared to multi-layer perceptron (MLP) or physics-informed neural networks (PINN). Various numerical experiments that cover computer science and engineering domains demonstrate that INN can solve over Zetta scale (10^{21}) partial differential equations and train/calibrate a dataset with extraordinary accuracy but fewer parameters using only a single graphics processing unit (GPU).
Comment: 9 pages, 2 figures