The Effect of Contrastive Pretraining on Downstream Tasks in Optical Remote Sensing

Jonathan Prexl, Michael Schmitt

Abstract: In this work, we investigate two critical design decisions that arise when adapting the concept of contrastive learning to optical earth observation data. We work within the framework of SimCLR in order to pre-train neural network architectures and test their respective applicability on downstream datasets over various common remote sensing tasks. During the training, we focus in detail on the concept of the creation of positive and negative pairs due to the introduction of different batch sampling strategies and color-related augmentations. We report all performance metrics as a function of the available training data and discuss underlying mechanisms in order to drive the understanding of contrastive learning in optical Earth observation forward.