Salient Object Detection on 360° Omnidirectional Image with Bi-Branch Hybrid Projection Network

Jie Zhang, Qiudan Zhang, Xuelin Shen, Xu Wang

Abstract: With the advent of panoramic cameras, modeling saliency in 360° omnidirectional images becomes very urgent and challenging. However, severe distortions limit the prediction accuracy of 360° saliency model. In this paper, we devise a bi-branch hybrid projection network (HPNet), which exploits characteristics of equirectangular projection (ERP) and cubic map projection (CMP) formats to predict salient objects in 360° omnidirectional images. Specifically, an ERP image and a CMP image are first fed into a bi-branch network to aggregate the comprehensive features of the omnidirectional image. Subsequently, to explore the coherence among ERP and CMP images, we design a hybrid projection feature fusion module to efficiently combine CMP and ERP features extracted from different layers. Ultimately, a progressive prediction module is developed to refine the features and locate salient objects incrementally, and then produce the final saliency map for the 360° omnidirectional image. Experimental results illustrate that our model is superior to the existing advanced methods in two publicly available datasets.