On Rotational Symmetry in the Loss landscape of Self-Supervised LearningDownload PDF

26 Sept 2022, 12:09 (modified: 09 Nov 2022, 02:12)NeurReps 2022 PosterReaders: Everyone
Keywords: self-supervised learning, collapse, symmetry breaking, phase transition
TL;DR: We analytically solve the loss landscape of self-supervised learning and identify the causes of complete and dimensional collapse to be relevant to symmetry breaking
Abstract: We derive an analytically tractable theory of SSL landscape and show that it accurately captures an array of collapse phenomena and identifies their causes.
4 Replies