OS3Flow: Optical and SAR Image Registration using Symmetry-guided Semi-dense Optical Flow

Zixuan Sun, Shuaifeng Zhi, Xuecong Liu, Weidong Jiang, Yongxiang Liu

Abstract: Abstract—Registration of optical and synthetic aperture radar(SAR) image pairs is a fundamental task in various remote sensing applications, including image fusion, target localization, and object detection. Unlike homogeneous image pairs, optical and SAR image pairs exhibit a significant modality gap, making it exceptionally challenging to extract consistent and reliable features. Particularly for optical and SAR image pairs with substantial geometric differences, few methods can achieve high-precision registration. To address this challenging task, we intro-duce a novel registration framework, called OS3Flow, leveraging on the implicit symmetry between heterogeneous image pairs to extract high-quality semi-dense flow estimations. We start by training the network in a multi-task manner using a standard flow regression loss as well as a symmetry loss with reverse input order. A confidence mask thus can be generated to measure the similarity between predictions at inference time. We then perform a linear regression upon selected flows with high confidence to estimate the parameters of underlying affine transformation. Experimental results on the public OS dataset and OXS-SAROPT dataset demonstrate that our proposed method can achieve superior accuracy and greater robustness compared to state-of-the-art methods, especially under large transformations.