Vectorized Conditional Neural Fields: A Framework for Solving Time-dependent PDEs
Keywords: Partial Differential Equations, PDEs, Implicit Neural Representations, INRs, Continuous Models, Neural Operators, Conditional Neural Fields, Transformers, Superresolution, Vision Transformers, Linear Transformers
TL;DR: Utilizing linear transformers, we propose a conditional neural fields based framework for solving time-dependent parametric partial differential equations that is fast and generalizes well to unseen PDE parameters.
Abstract: Transformer models are increasingly used for solving Partial Differential Equations (PDEs). However, they lack at least one of several desirable properties of an ideal surrogate model such as (i) generalization to PDE parameters not seen during training, (ii) spatial and temporal zero-shot super-resolution, (iii) continuous temporal extrapolation, (iv) applicability to PDEs of different dimensionalities, and (v) efficient inference for longer temporal rollouts. To address these limitations, we propose Vectorized Conditional Neural Fields (VCNeFs) which represent the solution of time-dependent PDEs as neural fields. Contrary to prior methods, VCNeFs compute, for a set of multiple spatio-temporal query points, their solutions in parallel while also modeling their dependencies through attention mechanisms. Moreover, VCNeF can condition the neural field on both the initial conditions and the parameters of the PDEs. An extensive set of experiments demonstrates that VCNeFs are competitive with and often outperform existing ML-based surrogate models.
Submission Number: 55
Loading