Sub-Sequential Physics-Informed Learning with State Space Model

Chenhui Xu, Dancheng Liu, Yuting Hu, Jiajie Li, Ruiyang Qin, Qingxiao Zheng, Jinjun Xiong

TL;DR: This paper introduces PINNMamba, a novel state-space-based sub-sequence learning framework for physics-informed neural networks.

Abstract: Physics-Informed Neural Networks (PINNs) are a kind of deep-learning-based numerical solvers for partial differential equations (PDEs). Existing PINNs often suffer from failure modes of being unable to propagate patterns of initial conditions. We discover that these failure modes are caused by the simplicity bias of neural networks and the mismatch between PDE's continuity and PINN's discrete sampling. We reveal that the State Space Model (SSM) can be a continuous-discrete articulation allowing initial condition propagation, and that simplicity bias can be eliminated by aligning a sequence of moderate granularity. Accordingly, we propose PINNMamba, a novel framework that introduces sub-sequence modeling with SSM. Experimental results show that PINNMamba can reduce errors by up to 86.3\% compared with state-of-the-art architecture. Our code is available at Supplementary Material.

Lay Summary:

When scientists use today’s artificial intelligence tools to solve physics problems—like forecasting how heat moves through a material or how waves travel—they sometimes find that the computer’s answers gradually drift away from reality. Our study pinpoints two reasons for this: (1) the networks prefer unnaturally “smooth” solutions, and (2) they learn only from a handful of sample points and therefore fail to carry the correct patterns forward in time.

We introduce PINNMamba, a new AI framework that acts like a long-term memory for these simulations while still working quickly. It slices each problem into short, overlapping time snippets and equips the network with a built-in mechanism that continuously links the snippets together, so information from the starting conditions is preserved all the way through the calculation. In tests on several difficult physics equations, this strategy slashed errors by as much as 86 percent compared with the best previous designs, giving far more trustworthy results without extra data or hand-tuning.

Link To Code: https://github.com/miniHuiHui/PINNMamba

Primary Area: Applications->Chemistry, Physics, and Earth Sciences

Keywords: Physics-Informed Neural Networks, State Space Model

Submission Number: 7