OmniArch: Building Foundation Model for Scientific Computing

Tianyu Chen, Haoyi Zhou, Ying Li, Hao Wang, Chonghan Gao, Rongye Shi, Shanghang Zhang, Jianxin Li

TL;DR: We build the foundation model for PDE solvers with 1D-2D-3D united pre-training and exhibit zero-shot capability.

Abstract: Foundation models have revolutionized language modeling, while whether this success is replicated in scientific computing remains unexplored. We present OmniArch, the first prototype aiming at solving multi-scale and multi-physics scientific computing problems with physical alignment. We addressed all three challenges with one unified architecture. Its pre-training stage contains a Fourier Encoder-decoder fading out the disharmony across separated dimensions and a Transformer backbone integrating quantities through temporal dynamics, and the novel PDE-Aligner performs physics-informed fine-tuning under flexible conditions. As far as we know, we first conduct 1D-2D-3D united pre-training on the PDEBench, and it sets not only new performance benchmarks for 1D, 2D, and 3D PDEs but also demonstrates exceptional adaptability to new physics via in-context and zero-shot learning approaches, which supports realistic engineering applications and foresight physics discovery.

Lay Summary:

Scientific simulations power everything from weather forecasting to aircraft design, but traditional methods require specialized coding and supercomputers. We present OmniArch, the first AI foundation model that can solve diverse physics problems across 1D, 2D, and 3D simulations using a single system—like how language models understand diverse texts.

Our key innovation is teaching AI the "language of physics" through frequency-based learning (like musical notes for equations) and a special Physics-Aligner that ensures predictions obey real-world laws. Trained on 11 types of physics problems, OmniArch outperforms specialized AI tools while showing human-like adaptability—it can solve new physics problems with minimal examples (in-context learning) or even zero examples (zero-shot learning).

This breakthrough could democratize scientific computing, allowing engineers to simulate complex systems faster while maintaining accuracy. Future applications may accelerate climate modeling, energy research, and materials discovery.

Link To Code: https://openi.pcl.ac.cn/cty315/OmniArch

Primary Area: Applications->Chemistry, Physics, and Earth Sciences

Keywords: AI for Science, Partial Differential Equations(PDEs), Foundation Model

Submission Number: 988