Continual Self-Supervised Learning in Earth Observation with Embedding Regularization

Hamna Moieez, Valerio Marsocci, Simone Scardapane

Abstract: Continual Self-Supervised Learning (CSSL) is a promising approach for intelligent systems that address the challenge of learning in scenarios with limited data, mirroring real-world conditions. However, CSSL remains relatively unexplored, especially in the context of Earth Observation (EO). In this paper, we investigate the problem of CSSL in remote sensing (RS), focusing on leveraging satellite and aerial imagery to develop systems that can continuously adapt and learn with minimal human intervention in data preparation. Specifically, we tackle the task of semantic segmentation, which has diverse applications in RS. Building upon existing work in the domain, we propose a novel algorithm called Continual Barlow Twins with Embedding Regularizer (CBT-ER). To evaluate the effectiveness of our approach, we conduct experiments on three heterogeneous datasets (i.e. Potsdam, DFC2022, SEN12MS). To ensure robust experimentation, we vary the availability of data labels (10%, 100%) and compare our approach against different baselines, showing encouraging performance.