Backdoor Attacks on Bimodal Salient Object Detection with RGB-Thermal Data

Wen Yin, Bin Benjamin Zhu, Yulai Xie, Pan Zhou, Dan Feng

Abstract: RGB-Thermal Salient Object Detection (RGBT-SOD) plays a critical role in complex scene recognition applications, such as autonomous driving. However, security research in this domain is still in its infancy. This paper presents the first backdoor attack on RGBT-SOD systems, generating saliency maps on triggered inputs that depict non-existent salient objects chosen by the attacker or falsely mark an entire image as fully salient or entirely non-salient. We uncover that triggers have an influence range for generating non-existent salient objects, supported by a theoretical analysis. Extensive experiments show the effectiveness of our attack in both digital and physical-world scenarios. Notably, our dual-modality backdoor attack achieves an Attack Success Rate (ASR) of 86.72% with only five pairs of poisoned images in model training. After investigating potential countermeasures, we find them inadequate in mitigating our attacks, highlighting the urgent need for robust defenses against sophisticated backdoor attacks in RGBT-SOD systems.