Fourier Learning Machines: Nonharmonic Fourier-Based Neural Networks for Scientific Machine Learning

TMLR Paper5887 Authors

Abstract: We introduce the Fourier Learning Machine (FLM), a neural network (NN) architecture designed to represent a multidimensional nonharmonic Fourier series. The FLM uses a simple feedforward structure with cosine activation functions to learn the frequencies, amplitudes, and phase shifts of the series as trainable parameters. This design allows the model to create a problem--specific spectral basis adaptable to both periodic and nonperiodic functions. Unlike previous Fourier--inspired NN models, the FLM is the first architecture able to represent a multidimensional Fourier series with a complete set of basis functions in separable form, doing so by using a standard Multilayer Perceptron--like architecture. A one--to--one correspondence between the Fourier coefficients and amplitudes and phase-shifts is demonstrated, allowing for the translation between a full, separable basis form and the cosine phase--shifted one. Additionally, we evaluate the performance of FLMs on several scientific computing problems, including benchmark Partial Differential Equations (PDEs) and a family of Optimal Control Problems (OCPs). Computational experiments show that the performance of FLMs is comparable, and often superior, to that of established architectures like SIREN and vanilla feedforward NNs.

Submission Length: Long submission (more than 12 pages of main content)

Changes Since Last Submission:

This revised version includes all of the major/critical corrections suggested by the reviewers. We also have corrected the identified typos and added some of the suggested experiments and discussion improvements. In particular, we have:

- Added the inviscid Burgers' PDE in the numerical experiments.
- Added the five--dimensional optimal control problem extending the RPS to the RPS--Spock--Lizard odd circulant game.
- An appendix containing a simple time series imputation experiment was added to showcase the potential of FLMs competing against the state-of-the-art models.
- We have reorganized the plots and tables containing the numerical results for the PDEs for better exposition and readability.
- We have added one figure containing the FLM with three inputs that illustrates the relationship of its architecture with the Lexi Signed matrix with the goal of better explaining how this matrix is constructed and how its rows define the signs of the frequency components in the cosine phase--shifted Fourier series representation.
- A table with the calculation of the total number of trainable parameters in each model for solving the PDEs is presented in appendix C.

Assigned Action Editor: ~William_T_Redman1

Submission Number: 5887