Accelerating Data Generation for Neural Operators via Krylov Subspace Recycling

Hong Wang, Zhongkai Hao, Jie Wang, Zijie Geng, Zhen Wang, Bin Li, Feng Wu

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Keywords: AI4PDE; Neural Operator; Data Generation; Krylov Subspace

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.

TL;DR: This paper proposes a novel method, namely SKR, to boost the data generation efficiency for neural operator training based on Krylov subspace recycling.

Abstract:

Learning neural operators for solving partial differential equations (PDEs) has attracted great attention due to its high inference efficiency.
However, training such operators requires generating a substantial amount of labeled data, i.e., PDE problems together with their solutions.
The data generation process is exceptionally time-consuming, as it involves solving numerous systems of linear equations to obtain numerical solutions to the PDEs.
Many existing methods solve these systems independently without considering their inherent similarities, resulting in extremely redundant computations.
To tackle this problem, we propose a novel method, namely **S**orting **K**rylov **R**ecycling (**SKR**), to boost the efficiency of solving these systems, thus significantly accelerating data generation for neural operators training.
To the best of our knowledge, SKR is the first attempt to address the time-consuming nature of data generation for learning neural operators.
The working horse of SKR is Krylov subspace recycling, a powerful technique for solving a series of interrelated systems by leveraging their inherent similarities.
Specifically, SKR employs a sorting algorithm to arrange these systems in a sequence, where adjacent systems exhibit high similarities.
Then it equips a solver with Krylov subspace recycling to solve the systems sequentially instead of independently, thus effectively enhancing the solving efficiency.
Both theoretical analysis and extensive experiments demonstrate that SKR can significantly accelerate neural operator data generation, achieving a remarkable speedup of up to 13.9 times.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Primary Area: applications to physical sciences (physics, chemistry, biology, etc.)

Submission Number: 7242