Self-Supervised Collaborative Distillation: Enhancing Lighting Robustness and 3D Awareness

Wonjun Jo, Hyunwoo Ha, Kim Ji-Yeon, Hawook Jeong, Tae-Hyun Oh

Keywords: Self-Supervised Learning, Representation Learning

TL;DR: We propose a novel self-supervised approach, Collaborative Distillation, which improves light-invariance and 3D awareness in 2D image encoders while retaining semantic context, integrating the strengths of 2D image and 3D LiDAR data.

Abstract: As deep learning continues to advance, self-supervised learning has made considerable strides. It allows 2D image encoders to extract useful features for various downstream tasks, including those related to vision-based systems. Nevertheless, pre-trained 2D image encoders face two key challenges: nighttime lighting conditions and limited 3D awareness, which are required for robust perception and 3D understanding of reliable vision-based systems. To address these issues, we propose a novel self-supervised approach, Collaborative Distillation, which improves light-invariance and 3D awareness in 2D image encoders while retaining semantic context, integrating the strengths of 2D image and 3D LiDAR data. Our method significantly outperforms competing methods in various downstream tasks across diverse lighting conditions and exhibits strong generalization ability. This advancement highlights our method's practicality and adaptability in real-world scenarios.

Submission Number: 37