1. Low dimensional approximation and generalization of multivariate functions on smooth manifolds using deep ReLU neural networks.
- Author
-
Labate, Demetrio and Shi, Ji
- Subjects
- *
ARTIFICIAL neural networks , *SMOOTHNESS of functions , *APPROXIMATION error , *GENERALIZATION , *PROBLEM solving - Abstract
The expressive power of deep neural networks is manifested by their remarkable ability to approximate multivariate functions in a way that appears to overcome the curse of dimensionality. This ability is exemplified by their success in solving high-dimensional problems where traditional numerical solvers fail due to their limitations in accurately representing high-dimensional structures. To provide a theoretical framework for explaining this phenomenon, we analyze the approximation of Hölder functions defined on a d -dimensional smooth manifold M embedded in R D , with d ≪ D , using deep neural networks. We prove that the uniform convergence estimates of the approximation and generalization errors by deep neural networks with ReLU activation functions do not depend on the ambient dimension D of the function but only on its lower manifold dimension d , in a precise sense. Our result improves existing results from the literature where approximation and generalization errors were shown to depend weakly on D. • Deep ReLU neural networks exhibit remarkable approximation properties for multivariate functions. • We prove that the approximation error of manifold-embedded data depends only on the manifold dimension. • Argument relies on preservation of pairwise ambient distances on the manifold. • Our result improves existing results where the approximation error depends weakly on the ambient dimension. • Our result implies an improved estimate of the generalization error. [ABSTRACT FROM AUTHOR]
- Published
- 2024
- Full Text
- View/download PDF