Continuous Reconstruction of Spatiotemporal Dynamical System with Neural PDE

Haonan Jia, Tao Zou, Bowen Du, Junchen Ye

Keywords: spatiotemporal reconstruction, Neural ODE, generative pre-trained model

Abstract: Recent studies have shown great promise in applying neural ordinary differential equations (ODEs) for spatiotemporal dynamical systems reconstruction, and the continuous modeling capability is naturally applied to capture the temporal evolution. However, we notice that few studies have employed ODE for spatial relationship modeling. Most of them utilized discrete models, such as CNNs or GNNs, ignoring the inherent continuity in spatial domain. To fill this gap, we propose Neural Partial Differential Equation (Neural PDE) to model spatiotemporal dynamical systems continuously and simultaneously by fitting the higher-order partial derivatives of the hidden states with respect to the spatial and temporal dimensions. Meanwhile, it is combined with a generative pretraining framework to dynamically optimize the initial latent states using an auto-decoder. For efficient training, we propose the overlapping multiple shooting, which splits long trajectories into overlapping sub-segments and enforces smooth transitions using continuity loss. Experiments show that Neural PDE achieves SOTA performance on both advection equation and Burgers equation, e.g., 74.29\% RMSE reduction compared to the second-best method under 1\% extremely sparse observations, and performs robustly in different resolutions and tasks generalization, which verifies the superiority of the continuous modeling and generalization capabilities.

Primary Area: learning on time series and dynamical systems

Submission Number: 6994