Deep Multi-Spectral Registration Using Invariant Descriptor Learning

Nati Ofir, Shai Silberstein, Hila Levi, Dani Rozenbaum, Yosi Keller, Sharon Duvdevani Bar

Abstract: In this work, we propose a deep-learning approach for aligning cross-spectral images. Our approach utilizes a learned descriptor invariant to different spectra. Multi-modal images of the same scene capture different characteristics and therefore their registration is challenging. To that end, we developed a feature-based approach for registering visible (VIS) to Near-Infra-Red (NIR) images. Our scheme detects corners by Harris and matches them by a patch-metric learned on top of a network trained using the CIFAR-10 dataset. As our experiments demonstrate, we achieve accurate alignment of cross-spectral images with sub-pixel accuracy. Comparing to contemporary state-of-the-art, our approach is more accurate in the task of VIS to NIR registration.