Hierarchical Uncertainty-Aware Salient Object Detection for $360 ^{\circ }$ Images via Bi-Projection Collaborative Learning

Qiudan Zhang, Kaiyu Ji, Jie Zhang, Xu Wang, Zhaoqing Pan, Jianmin Jiang

Abstract: In this article, we develop a hierarchical uncertainty-aware $360 ^{\circ }$ image salient object detection methodology that explicitly explores the geometric and spatial complementary coherence of Tangent projection (TP) and Equirectangular projection (ERP) by a collaborative learning strategy. Concretely, to mitigate spherical distortion, we first intend to learn saliency-related features from less-distorted tangent images, in which a deformation-aware attention block is introduced to mitigate the geometric distortion caused by projecting a $360 ^{\circ }$ image onto a 2D plane. However, the discrepancies among tangent images pose a new challenge to $360 ^{\circ }$ image salient object detection. To tackle this issue and achieve accurate localization for salient objects of all sizes, we design a spatial-frequency saliency feature aggregation module to leverage fast Fourier convolution to capture global contextual information from ERP images, such that obtaining more representative saliency features. Moreover, a hierarchical uncertainty-aware bi-projection consistency learning module with strong local-global information embedding capabilities is constructed, which learns the geometric and spatial correlations between tangent images and ERP images via a collaborative learning strategy. Ultimately, salient object maps are produced for $360 ^{\circ }$ images on the basis of the merged saliency features driven by the uncertainty. Extensive experiments show that our developed method improves ${\mathrm{F}}_\beta ^{\sigma }$ by an average of 31.67% compared to twenty existing advanced methods on the publicly available 360-SOD dataset.

External IDs:dblp:journals/tmm/ZhangJZWPJ25