Sensor Parameter Encoding for Multi-Sensor Self-Supervised Learning via Masked Autoencoders

Jonathan Prexl, Michael Schmitt

Abstract: This study presents a novel pretraining approach for self-supervised learning on optical Earth observation satellite data based on the masked autoencoder paradigm. Unlike typical methods limited to a single sensor’s data, our method operates across various sensors by encoding physical sensor parameters into the learning step, to account for the unique differences among sensor designs. This enables merging training datasets acquired with different sensors as well as performing inference in a sensor-independent manner. Successful encoding of the sensor parameters through our approach is shown through testing on a downstream land-cover mapping task, where baseline models are outperformed by up to 6 points for the F1-score.