From Cheap Geometry to Expensive Physics: A Physics-agnostic Pretraining Framework for Neural Operators

Zhizhou Zhang, Youjia Wu, Kaixuan Zhang, Yanjia Wang

Keywords: Neural Operator, Pretraining, Physics-agnostic, Partial Differential Equation, Autoencoder

Abstract: Industrial design evaluation often relies on high-fidelity simulations of governing partial differential equations (PDEs). While accurate, these simulations are computationally expensive, making dense exploration of design spaces impractical. Operator learning has emerged as a promising surrogate for high-fidelity physical simulations, enabling fast prediction of partial differential equation (PDE) solutions. However, accuracy of neural operators is largely affected by the amount of training data, which must be generated by expensive numerical solvers. In practical industrial scenarios, there exists large collections of candidate geometries remain unsolved due to the high computation cost. These geometry-only samples contain no physical field labels and are therefore ignored in standard operator learning pipelines. In this work, we propose a general physics-agnostic pretraining framework to exploit this abundant geometric resource to improve the performance of neural operators. Specifically, we pretrain an autoencoder on a self-supervised proxy task to reconstruct geometry (e.g., via occupancy), learning an expressive latent representation without PDE supervision. Neural operators then leverage the pretrained latent embedding to learn more effectively from limited physics labels. An error decomposition analysis is provided to help understand the effectiveness of the physics-agnostic pretraining framework. Across four PDE datasets and three state-of-the-art transformer-based neural operators, our pretraining strategy consistently improves prediction accuracy. These results demonstrate that representations from physics-agnostic pretraining provide a powerful foundation for data-efficient operator learning. The code is publicly available at: https://github.com/zzzwoniu/Physics-agnostic-Operator-Pretraining.git

Primary Area: applications to physical sciences (physics, chemistry, biology, etc.)

Submission Number: 14824