When Neural PDE Solvers Fail to Extrapolate: Spectral Truncation and Boundary Condition Neglect

Rahul D Ray

Keywords: neural PDE solvers, Fourier neural operators, physics-informed neural networks, spectral truncation, boundary conditions, extrapolation, generalization, partial differential equations, operator learning, spectral methods

TL;DR: Neural PDE solvers interpolate well but extrapolate poorly: FNOs lack high-frequency modes, PINNs decouple boundaries from interiors, successful training does not imply a learned solution operator.

Abstract: Neural solvers for parametric partial differential equations achieve strong empirical accuracy within trained regimes, yet their ability to learn true solution operators remains unclear. We present a controlled diagnostic study of two widely used approaches, the Fourier Neural Operator and Physics Informed Neural Networks, designed to isolate structural extrapolation failures independent of numerical error or optimization instability. For the two dimensional Helmholtz equation, we show that the Fourier Neural Operator generalizes to lower frequency regimes but fails under high frequency extrapolation due to fixed spectral truncation, with prediction errors concentrating in unresolved Fourier modes. For a parametrically scaled Poisson problem, we demonstrate that Physics Informed Neural Networks trained at a single parameter value fail to recover correct amplitude scaling under boundary condition variation. Boundary errors remain small while interior solution errors grow substantially, revealing boundary interior decoupling induced by loss imbalance. Together, these results show that strong in distribution performance can mask reduction of the true operator to a lower capacity surrogate aligned with the training distribution. Our findings highlight the need for diagnostic extrapolation benchmarks and architectures explicitly designed for spectral and parametric generalization.

Submission Number: 41