Algebraic priors for approximately equivariant networks

Riccardo Ali, Pietro Lio, Jamie Vicary

Track: long paper (up to 8 pages)

Keywords: approximate equivariance, representation theory, symmetry

TL;DR: We prove that an approximately equivariant encoder's latent space must contain the regular representation, and enforce this algebraic prior with a lightweight loss without additional learnable parameters to achieve SOTA.

Abstract:

Equivariant neural networks incorporate symmetries through group actions, embedding them as an inductive bias to improve performance. Existing methods learn an equivariant action on the latent space, or design architectures that are equivariant by construction. These approaches often deliver strong empirical results but can involve architecture-specific constraints, large parameter counts, and high computational cost.
We challenge the paradigm of complex equivariant architectures with a parameter-free approach grounded in group representation theory. We prove that for an equivariant encoder over a finite group, the latent space must almost surely contain one copy of its regular representation for each linearly independent data orbit, which we explore with a number of empirical studies. Leveraging this foundational algebraic insight, we impose the group's regular representation as an inductive bias via an auxiliary loss, adding no learnable parameters. Our extensive evaluation shows that this method matches or outperforms specialized models in several cases, even those for infinite groups.
We further validate our choice of the regular representation through an ablation study, showing it consistently outperforms defining and trivial group representation baselines.

Anonymization: This submission has been anonymized for double-blind review via the removal of identifying information such as names, affiliations, and identifying URLs.

Presenter: ~Riccardo_Ali1

Format: Maybe: the presenting author will attend in person, contingent on other factors that still need to be determined (e.g., visa, funding).

Funding: Yes, the presenting author of this submission falls under ICLR’s funding aims, and funding would significantly impact their ability to attend the workshop in person.

Serve As Reviewer: ~Riccardo_Ali1

Submission Number: 98