Permutation Equivariant Neural Networks for Symmetric Tensors

Edward Pearce-Crump

Abstract:

Incorporating permutation equivariance into neural networks has proven to be useful in ensuring that models respect symmetries that exist in
data. Symmetric tensors, which naturally appear
in statistics, machine learning, and graph theory,
are essential for many applications in physics,
chemistry, and materials science, amongst others. However, existing research on permutation
equivariant models has not explored symmetric
tensors as inputs, and most prior work on learning
from these tensors has focused on equivariance
to Euclidean groups. In this paper, we present
two different characterisations of all linear permutation equivariant functions between symmetric
power spaces of $\mathbb{R}^{n}$. We show on two tasks that
these functions are highly data efficient compared
to standard MLPs and have potential to generalise
well to symmetric tensors of different sizes.

Lay Summary:

Many scientific fields use data in the form of symmetric tensors, but current machine learning models do not fully take advantage of their natural symmetries. This limits how well models can learn from such data.
We developed a new way to design models that respect these symmetries by fully characterising all linear functions between symmetric tensors that are equivariant under permutations. To make this characterisation practical, we introduced a method that represents these functions without needing large amounts of memory to store weight matrices, making it adaptable to symmetric tensors of different sizes.
Having been tested on two example problems, our method learns from less data and generalises better than standard neural networks to data that can be represented in the form of symmetric tensors. This opens up new possibilities for applying machine learning to scientific domains where symmetric tensors play a central role, such as physics, chemistry, and materials science.

Primary Area: Theory->Deep Learning

Keywords: Equivariance, deep learning, permutations, symmetric tensors

Submission Number: 3265