Modeling Training Dynamics and Error Estimates of DNN-based PDE Solvers: A Continuous Framework

ICLR 2026 Conference Submission16448 Authors

Keywords: DNN-based PDE solvers, SGD, continuous modeling, error estimates

Abstract:

Deep neural network-based PDE solvers have shown remarkable promise for tackling high-dimensional partial differential equations, yet their training dynamics and error behavior are not well understood. This paper develops a unified continuous-time framework based on stochastic differential equations to analyze the noisy regularized stochastic gradient descent algorithm when applied to deep PDE solvers.
Our approach establishes weak error between this algorithm and its continuous approximation, and provides new asymptotic error characterizations via invariant measures. Importantly, we overcome the restrictive global Lipschitz continuity assumption, making our theory more applicable to practical deep networks. Furthermore, we conduct systematic experiments to reveal how stochasticity affects solution accuracy and the stability domains of optimizers. These results provide actionable guidance for practical training, particularly showing that adaptively switching optimizers and step sizes in the presence of different local minima during training can be beneficial for neural PDE solvers.

Primary Area: other topics in machine learning (i.e., none of the above)

Submission Number: 16448