Mixture of Neural Operators: Incorporating Historical Information for Longer Rollouts
Keywords: Neural PDE Solvers, Neural Operators, Autoregressive Methods
TL;DR: Incorporating historical information in neural PDE solvers frequently worsens performance, we introduce a framework to address this issue.
Abstract: Traditional numerical solvers for time-dependent partial differential equations (PDEs) notoriously require high computational resources and necessitate recomputation when faced with new problem parameters. In recent years, neural surrogates have shown great potential to overcome these limitations. However, it has been paradoxically observed that incorporating historical information into neural surrogates worsens their rollout performance. Drawing inspiration from multistep methods that use historical information from previous steps to obtain higher-order accuracy, we introduce the Mixture of Neural Operators (MoNO) framework; a collection of neural operators, each dedicated to processing information from a distinct previous step. We validate MoNO on the Kuramoto-Sivashinsky equation, demonstrating enhanced accuracy and stability of longer rollouts, greatly outperforming neural operators that discard historical information.
Submission Number: 62
Loading