Generalized Fourier Features for Coordinate-Based Learning of Functions on Manifolds

Carlos Esteves, Tianjian Lu, Mohammed Suhail, Yi-fan Chen‎, Ameesh Makadia

Keywords: positional encoding, fourier features, coordinate-based mlp

Abstract: Recently, positional encoding of input coordinates has been found crucial to enable learning of high-frequency functions with multilayer perceptrons taking low-dimensional coordinate values. In this setting, sinusoids are typically used as a basis for the encoding, which is commonly referred to as "Fourier Features". However, using sinusoids as a basis assumes that the input coordinates lie on Euclidean space. In this work, we generalize positional encoding with Fourier features to non-Euclidean manifolds. We find appropriate bases for positional encoding on manifolds through generalizations of Fourier series. By ensuring the encodings lie on a hypersphere and that the appropriate shifts on the manifold preserve inner-products between encodings, our model approximates convolutions on the manifold, according to the neural tangent kernel (NTK) assumptions. We demonstrate our method on various tasks on different manifolds: 1) learning panoramas on the sphere, 2) learning probability distributions on the rotation manifold, 3) learning neural radiance fields on the product of cube and sphere, and 4) learning light fields represented as the product of spheres.

One-sentence Summary: We generalize the positional encoding method using Fourier features for coordinate-based learning of functions on non-Euclidean manifolds.