Sharp Approximation Rates for Neural Operators on Sobolev Spaces: Bridging the Gap Between Theory and Practice

Mysore supreeth

Keywords: Neural operators, Fourier Neural Operator, DeepONet, Sobolev spaces, approximation theory, PDEs, Green's function, minimax optimality, operator learning

TL;DR: Sharp $O(K^{-s} + W^{-2/d} + L^{-1})$ rates for FNO; minimax optimal and matches best linear method; predicted scaling matches empirical error within 5\%.

Abstract: Neural operators (Fourier Neural Operator, DeepONet) have achieved remarkable empirical success in learning solution operators of partial differential equations (PDEs), yet their approximation theory remains incomplete. We establish sharp approximation rates for neural operators mapping between Sobolev spaces, resolving the long-standing gap between known upper bounds and empirical performance. For the Fourier Neural Operator with $K$ modes, $L$ layers, and width $W$, we prove an upper bound of $O(K^{-s} + W^{-2/d} + L^{-1})$ for approximating $s$-regular operators, where $d$ is the input dimension. We complement this with a matching lower bound showing that any continuous neural operator architecture requires $\Omega(K^{-s})$ modes to achieve comparable rates, proving our bounds are minimax optimal. As a consequence, we demonstrate that FNO achieves the same approximation rate as the best linear method (truncated SVD of the Green's function) up to logarithmic factors, settling the fundamental question of whether nonlinearity provides an advantage for operator learning. We validate our theory on Darcy flow, Navier-Stokes, and advection equations, demonstrating that the theoretically predicted scaling laws $K^{-s}$ match empirical error decay within 5\% across all benchmarks. Our results provide practitioners with the first rigorous guidelines for architecture selection and hyperparameter tuning in neural operator design.

Submission Number: 153