OSFlow: Optical and SAR Image Registration Using Symmetry-Guided Semi-Dense Optical Flow

Zixuan Sun, Shuaifeng Zhi, Kai Huo, Xuecong Liu, Weidong Jiang, Yongxiang Liu

Abstract: Registration of optical and synthetic aperture radar (SAR) image pairs is a fundamental task in various remote sensing applications, including image fusion, target localization, and object detection. Unlike homogeneous image pairs, optical and SAR image pairs exhibit a significant modality gap, making it exceptionally challenging to extract consistent and reliable features. Particularly for optical and SAR image pairs with substantial geometric differences, few methods can achieve high-precision registration. To address this challenging task, we introduce a novel registration framework, called OS3Flow, leveraging on the implicit symmetry between heterogeneous image pairs to extract high-quality semi-dense flow estimations. We start by training the network in a multitask manner using a standard flow regression loss as well as a symmetry loss with reverse input order. A confidence mask thus can be generated to measure the similarity between predictions at inference time. We then perform a linear regression upon selected flows with high confidence to estimate the parameters of underlying affine transformation. Under large transformations, our proposed method achieves an average registration error of less than three pixels on the public OS dataset and Wuhan University-optical (WHU-OPT)-SAR dataset, demonstrating superior accuracy and robustness compared to state-of-the-art methods.