Learning Group Actions on Latent Representations

Yinzhu Jin, Aman Shrivastava, Tom Fletcher

Keywords: group action, representation learning, image rendering

TL;DR: A model to learn group action on latent factors, e.g. manipulating objects on 2D images.

Abstract: In this work, we introduce a new approach to model group actions in autoencoders. Diverging from prior research in this domain, we propose to learn the group actions on the latent space rather than strictly on the data space. This adaptation enhances the versatility of our model, enabling it to learn a broader range of scenarios prevalent in the real world, where groups can act on latent factors. Our method allows a wide flexibility in the encoder and decoder architectures and does not require group-specific layers. In addition, we show that our model theoretically serves as a superset of methods that learn group actions on the data space. We test our approach on five image datasets with diverse groups acting on them and demonstrate superior performance to recently proposed methods for modeling group actions.

Primary Area: Deep learning architectures

Submission Number: 21500