Uniform Approximation of Equivariant/Invariant Neural Networks

Jianchuan Huang, Fengxiang He, Bohan Wang, Xiliang Lu, Dacheng Tao

Primary Area: learning theory

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Keywords: Approximation, Equivariant neural networks, Invariant neural networks

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.

TL;DR: In this paper, the approximations of equivariant neural networks under general groups are introduced.

Abstract: Equivariant structures have been widely adopted in graph neural networks due to their demonstrated effectiveness in various machine learning tasks, such as point clouds, biology, and chemistry. We focuses on investigating the approximation power of equivariant neural networks. Specifically, we prove that equivariant neural networks with any continuous activation function can approximate any continuous equivariant function. Our theorem is established based on a novel composition of any subgroup $G$ of a symmetric group $S_M$. Additionally, we note that this representation may not work for certain invariant continuous functions when the dimension of the latent space is smaller than the dimension of the input space.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 1404