Sign and Basis Invariant Networks for Spectral Graph Representation Learning

Derek Lim, Joshua David Robinson, Lingxiao Zhao, Tess Smidt, Suvrit Sra, Haggai Maron, Stefanie Jegelka

Keywords: Invariance, Equivariance, Graph Neural Networks, Eigenvectors, Spectral

TL;DR: We develop neural networks invariant to the symmetries of eigenvectors, which are theoretically expressive and empirically improve graph neural networks and other models.

Abstract: We introduce SignNet and BasisNet---new neural architectures that are invariant to two key symmetries displayed by eigenvectors: (i) sign flips, since if v is an eigenvector then so is -v; and (ii) more general basis symmetries, which occur in higher dimensional eigenspaces with infinitely many choices of basis eigenvectors. We prove that our networks are universal, i.e., they can approximate any continuous function of eigenvectors with the desired invariances. Moreover, when used with Laplacian eigenvectors, our architectures are provably expressive for graph representation learning: they can approximate any spectral graph convolution, can compute spectral invariants that go beyond message passing neural networks, and can provably simulate previously proposed graph positional encodings. Experiments show the strength of our networks for molecular graph regression, learning expressive graph representations, and learning neural fields on triangle meshes.

Community Implementations: [ 3 code implementations](https://www.catalyzex.com/paper/sign-and-basis-invariant-networks-for/code)