Refined generalization analysis of the Deep Ritz Method and Physics-Informed Neural Networks

Xianliang Xu, Ye Li, Zhongyi Huang

TL;DR: In this paper, we present refined generalization bounds for the Deep Ritz Method (DRM) and Physics-Informed Neural Networks (PINNs).

Abstract:

In this paper, we derive refined generalization bounds for the Deep Ritz Method (DRM) and Physics-Informed Neural Networks (PINNs). For the DRM, we focus on two prototype elliptic partial differential equations (PDEs): Poisson equation and static Schrödinger equation on the
$d$-dimensional unit hypercube with the Neumann boundary condition. Furthermore, sharper generalization bounds are derived based on the localization techniques under the assumptions that the exact solutions of the PDEs lie in the Barron spaces or the general Sobolev spaces. For the PINNs, we investigate the general linear second order elliptic PDEs with Dirichlet boundary condition using the local Rademacher complexity in the multi-task learning setting. Finally, we discuss the generalization error in the setting of over-parameterization when solutions of PDEs belong to Barron space.

Lay Summary:

(1) Neural network-based methods for solving PDEs have gained popularity in scientific computing, yet their theoretical foundations remain incompletely understood.

(2) Under the scenario where PDE solutions belong to Barron spaces and Sobolev spaces, we employ localization techniques to derive sharper generalization bounds for two popular methods: Physics-Informed Neural Networks (PINNs) and Deep Ritz Method (DRM).

(3) This will advance the application of machine learning theory to a wide range of problems in scientific computing.

Primary Area: Theory->Learning Theory

Keywords: Deep Ritz Method, Physics-Informed Neural Networks, Fast Rates, Generalization Analysis

Submission Number: 4398