Optical-to-SAR image registration using a combination of CNN descriptors and cross-correlation coefficient

Akhmedkhan Shabanov, Sergey Gladilin, Evgeny A. Shvets

Abstract: Image registration is a problem of aligning two or more images of the same scene or object. The case when images are taken using different sensors - multimodal image registration - has applications in medical imaging and remote sensing. Unfortunately, many of the existing image registration methods operate under crude assumptions (i.e., the intensities of images are linearly correlated), which makes them inapplicable for the accurate multimodal registration. One approach to this task is to use deep learning to capture the complex intensity dependencies between images of different modalities. However, while deep learning methods produce good results, most of them are trained end-to-end and do not utilize the accumulated body of knowledge about image registration using “classic” information-theoretic and statistical methods. In this paper we consider the specific case of multimodal image registration - of optical and synthetic aperture radar (SAR) images. We use classic feature-based registration pipeline (first, corresponding feature points are found, then RANSAC is used as the transform estimator). Within this method we compare the effectiveness of various feature point detection and correspondence methods - both neural network-based and traditional. We find that Siamese network outperforms (but only slightly) the classic cross-entropy-based method for finding correspondences. Finally, we propose a hybrid method and show that it outperforms both “classic” method and an end-to-end network by a significant margin.