Geometric Contrastive Learning

Published: 31 Jul 2023, Last Modified: 24 Aug 2023VIPriors 2023 OralPosterTBDEveryoneRevisionsBibTeX
Keywords: geometric deep learning, contrastive learning, representation learning, self-supervised learning
Abstract: Contrastive learning has been a long-standing research area due to its versatility and importance in learning representations. Recent works have shown improved results if the learned representations are constrained to be on a hypersphere. However, this prior geometric constraint is not fully utilized during training. In this work, we propose making use of geodesic distances on the hypersphere to learn contrasts between representations. Through empirical results, we show that this contrastive learning approach improves downstream tasks across different contrastive learning frameworks. We show that having geometric inductive priors perform even better in contrastive learning if used along with other correct geometric information.
Submission Number: 20