Exploring Dimensional Collapse in Self-Supervised Video Representation Learning

Paul Kapust, Monika Kwiatkowski, Olaf Hellwich, Patrik Reiske

Keywords: deep learning, self-supervised learning, dimensional collapse

Abstract: In the field of joint embedding methods, the complete collapse to a constant feature vector is a clear indication of an immediate deficiency in the approach. Another critical concern, known as dimensional collapse, describes the utilization of a feature space only to a lower-dimensional subspace. Despite extensive efforts to address complete collapse through various preventive strategies, dimensional collapse remains largely unexplored. This paper aims to bridge this gap by extending the examination of dimensional collapse to video representation learning. Our source code is publicly available.

Submission Number: 199