Geometric Contrastive Learning

Yeskendir Koishekenov, Sharvaree Vadgama, Riccardo Valperga, Erik J Bekkers

Keywords: geometric deep learning, contrastive learning, representation learning, self-supervised learning

Abstract: Contrastive learning has been a long-standing research area due to its versatility and importance in learning representations. Recent works have shown improved results if the learned representations are constrained to be on a hypersphere. However, this prior geometric constraint is not fully utilized during training. In this work, we propose making use of geodesic distances on the hypersphere to learn contrasts between representations. Through empirical results, we show that this contrastive learning approach improves downstream tasks across different contrastive learning frameworks. We show that having geometric inductive priors perform even better in contrastive learning if used along with other correct geometric information.

Submission Number: 20