Reduced-rank Factorized Fourier Neural Operator

Chieh-An Chou, Lu-Hung Chen

Abstract:

We present R$^2$-FFNO, a novel neural operator architecture designed to address the overparameterization common in Factorized Fourier Neural Operators (FFNO) through reduced-rank factorization of spectral components. While neural operators are effective for learning solutions to partial differential equations (PDE), their architectures often contain an excessive number of parameters, which can lead to overfitting and diminished generalization capabilities. Inspired from reduced-rank learning techniques, the R$^2$-FFNO approach decomposes spectral kernels into lower-rank representations, enabling systematic control over the model's capacity. This low-rank factorization facilitates a balance between the model's expressiveness and capability of generalization. Empirical analysis reveals that performance saturates once an optimal rank is achieved and degrades if the rank is increased beyond this point. This observation highlights an optimal trade-off between model complexity and accuracy, underscoring the importance of principled rank selection in designing neural operators.

To further enhance performance, a targeted data augmentation strategy is utilized. This strategy introduces high-frequency variations during training to address spectral bias, thereby enhancing the model's capacity to resolve fine-scale PDE dynamics. A comprehensive evaluation on benchmark datasets confirms the efficacy of the R$^2$-FFNO. Compared to the original FFNO architecture, R$^2$-FFNO demonstrates significant error reductions: 46.5\% reduction in the Navier-Stokes problem, 31.6\% in the Kolmogorov Flow problem, and 34.7\% in the Darcy Flow problem. The proposed method offers a principled framework for managing overparameterization in neural operators, contributing to the development of more efficient and generalizable PDE solvers for wide application in scientific computing. Our code is available at https://github.com/Chieh997/R2FFNO.

Submission Number: 163