How important are specialized transforms in Neural Operators?
Keywords: Deep Learning, PDE solvers, Generalization, Transforms
TL;DR: Parameterized linear transforms generalize to a large class of PDEs and perform on-par with specialized transforms in Neural Operators.
Abstract: Transform-based Neural Operators like Fourier Neural Operators and Wavelet Neural Operators have received a lot of attention for their potential to provide fast solutions for systems of Partial Differential Equations. In this work, we investigate what could be the cost in performance, if all the transform layers are replaced by learnable linear layers. We observe that linear layers suffice to provide performance comparable to best-known transform-based layers and seem to do so at possibly a compute time advantage as well. We believe that this observation can have significant implications for future work on Neural Operators.
Submission Number: 68
Loading