Learning Physical Simulation with Message Passing Transformer
Keywords: Physical Simulation, Graph Neural Network, Learned Simulation, Message Passing
Abstract: Machine learning methods for mesh-based physical simulation have achieved significant success in recent years. We propose the Historical Message-Passing Integration Transformer (HMIT), an architecture based on Graph Neural Networks that incorporates a message passing framework and applies Graph Fourier Loss (GFL) for model optimization. To mitigate over-squashing, capture fine-grained details, and scale linearly with node count, we introduce Historical Message-Passing Attention (HMPA), which integrates multi-step historical message-passing information for each node with feature-wise softmax and employs a decoder-only architecture. Additionally, to modulate loss at specific frequencies and handle varying energy levels, we introduce GFL, which uses a frequency-domain energy adjustment schedule. To improve computational efficiency, we precompute the graph's Laplacian eigenvectors before training. Our architecture achieves significant accuracy improvements in shart- and long-term rollouts for both Lagrangian and Eulerian dynamical systems compared to current methods.
Primary Area: applications to physical sciences (physics, chemistry, biology, etc.)
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Submission Number: 2831
Loading