Out-of-distribution generalization of deep-learning surrogates for 2D PDE-generated dynamics in the small-data regime

Binh Duong Nguyen, Stefan Sandfeld

Keywords: surrogate model; out-of-distribution generalization; autoregressive

TL;DR: Our proposed deep-learning surrogates could learn the underlying mechanics of 2D PDE-generated dynamics without being informed by any physical information.

Abstract: Partial differential equations (PDEs) are a central tool for modeling the dynamics of physical, engineering, and materials systems, but high-fidelity simulations are often computationally expensive. At the same time, many scientific applications can be viewed as the evolution of spatially distributed fields, making data-driven forecasting of such fields a core task in scientific machine learning. In this work we study autoregressive deep-learning surrogates for two-dimensional PDE dynamics on periodic domains, focusing on generalization to out-of-distribution initial conditions within a fixed PDE and parameter regime and on strict small-data settings with at most $\mathcal{O}(10^2)$ simulated trajectories per system. We introduce a multi-channel U-Net with enforced periodic padding (me-UNet) that takes short sequences of past solution fields of a single representative scalar variable as input and predicts the next time increment. We evaluate me-UNet on various PDE families and compare it to ViT, AFNO, PDE-Transformer, and KAN-UNet under a common training setup. Across all datasets, me-UNet matches or outperforms these more complex architectures in terms of field-space error, spectral similarity, and physics-based metrics for in-distribution rollouts, while requiring substantially less training time, and also generalizes qualitatively to unseen initial conditions.

Journal Opt In: Yes, I want to participate in the IOP focus collection submission

Submission Number: 55