Almost Equivariance via Lie Algebra Convolutions
Submission Track: Extended Abstract
Keywords: equivariance, partial equivariance, almost equivariance, lie groups, lie algebras
Abstract: Recently, the $\textit{equivariance}$ of models with respect to a group action has
become an important topic of research in machine learning. Analysis of the built-in equivariance of
existing neural network architectures, as well as the study of methods for building model architectures that
explicitly ``bake in'' equivariance, have become significant research areas in their own right.
However, imbuing an architecture with a specific group equivariance imposes a strong prior on the types of data
transformations that the model expects to see. While strictly-equivariant models enforce symmetries, such
as those due to rotations or translations, real-world data does not always follow such strict equivariances,
be it due to noise in the data or underlying physical laws that encode only approximate or partial symmetries.
In such cases, the prior of strict equivariance can actually prove too strong and cause models
to underperform on real-world data. Therefore, in this work we study a closely related topic,
that of $\textit{almost equivariance}$. We give a practical method for encoding
almost equivariance in models by appealing to the Lie algebra of a Lie group and defining $\textit{Lie algebra convolutions}$.
We demonstrate that Lie algebra convolutions offer several benefits over Lie group convolutions,
including being computationally tractable and well-defined for non-compact groups.
Finally, we demonstrate the validity of our approach by benchmarking against datasets
in fully equivariant and almost equivariant settings.
Submission Number: 2
Loading