Towards a Physics Foundation Model

Florian Wiesner, Matthias Wessling, Stephen Baek

Keywords: Physics Foundation Model, Multi-physics Learning, In-context Learning, Zero-shot Generalization, Scientific Machine Learning, Physics-Aware Machine Learning, Spatiotemporal Transformers

TL;DR: We present GPhyT, a transformer trained on 1.8TB of diverse physics simulations that can zero-shot generalize to entirely new physical systems by inferring governing dynamics from context alone.

Abstract: Foundation models have revolutionized natural language processing through a ``train once, deploy anywhere'' paradigm, where a single pre-trained model adapts to countless downstream tasks without retraining. Access to a Physics Foundation Model (PFM) would be transformative - democratizing access to high-fidelity simulations, accelerating scientific discovery, and eliminating the need for specialized solver development. Yet current physics-aware machine learning approaches remain fundamentally limited to single, narrow domains and require retraining for each new system. We present the General Physics Transformer (GPhyT), trained on 1.8 TB of diverse simulation data, that demonstrates foundation model capabilities are achievable for physics. Our key insight is that transformers can learn to infer governing dynamics from context, enabling a single model to simulate fluid-solid interactions, shock waves, thermal convection, and multi-phase dynamics without being told the underlying equations. GPhyT achieves three critical breakthroughs: (1) superior performance across multiple physics domains, outperforming specialized architectures by more than 7x, (2) plausible zero-shot generalization to entirely unseen physical systems through in-context learning, and (3) more stable long-term predictions through long-horizon rollouts. By establishing that a single model can learn generalizable physical principles from data alone, this work opens the path toward a universal PFM that could transform computational science and engineering.

Primary Area: applications to physical sciences (physics, chemistry, biology, etc.)

Submission Number: 8602