Learning Interpretable Low-dimensional Representation via Physical Symmetry

Xuanjie Liu, Daniel Chin, Yichen Huang, Gus Xia

Keywords: Physics Symmetry, Time series data, Self-supervised Learning, Representation Augmentation

TL;DR: This study uses physics symmetry as an effective inductive bias to learn interpretable representations from time-series data in a self-supervised fashion.

Abstract: We have recently seen great progress in learning interpretable music representations, ranging from basic factors, such as pitch and timbre, to high-level concepts, such as chord and texture. However, most methods rely heavily on music domain knowledge. It remains an open question what general computational principles *give rise to* interpretable representations, especially low-dim factors that agree with human perception. In this study, we take inspiration from modern physics and use *physical symmetry* as a self-consistency constraint for the latent space. Specifically, it requires the prior model that characterises the dynamics of the latent states to be *equivariant* with respect to certain group transformations. We show that physical symmetry leads the model to learn a *linear* pitch factor from unlabelled monophonic music audio in a self-supervised fashion. In addition, the same methodology can be applied to computer vision, learning a 3D Cartesian space from videos of a simple moving object without labels. Furthermore, physical symmetry naturally leads to *counterfactual representation augmentation*, a new technique which improves sample efficiency.

Submission Number: 11097