4SCIG: A Four-Branch Framework to Reduce the Interference of Sky Area in Cross-View Image Geo-Localization

Jiangshan Li, Chunfang Yang, Baojun Qi, Ma Zhu, Nan Wu

Abstract: Cross-view image geo-localization is a technique that matches a query ground image with a geo-tagged satellite image. Due to the difference between ground and satellite views, the sky area frequently existing in the ground images is not possible to appear in the satellite images, which would interfere with the cross-view image matching. In this work, we argue that the sky area in the ground images would distract the feature and consequently reduce the accuracy of geo-localization. Therefore, we propose a four-branch framework to reduce the interference of sky area in cross-view image geo-localization (4SCIG), with two ground branches and two satellite branches. In two ground branches, the sky area in the ground image will be removed using two strategies. Meanwhile, in the two satellite branches, the satellite image would be aligned to ground-view by polar and projective transforms. Then, two sky-cropped ground images and two transformed satellite images will be input into the backbones of four branches, respectively. Finally, we design a multiple constraint loss (MCL) to optimize the four-branch framework. Extensive experiments on two standard datasets CVUSA and CVACT demonstrate that the proposed 4SCIG can significantly boost the geo-localization accuracy of previous methods.