PhysMCA: Physical Multi-modal Backdoor–Adversarial Example Collaborative Attack

Wenxin Liu, Xing Yang, Liquan Chen, Haoli Xu, Qiantao Yang, jun chen, Zhenhao Zhang

Keywords: Physical domain, backdoor attack, adversarial attack, collaborative attack

TL;DR: Multi-modal backdoor-adversarial collaborative attack framework for the physical domain

Abstract: The multi-modal visual system relies on deep learning models to enable all-weather perception, but it faces the dual security threats of backdoor and adversarial attacks. Although targeted defense methods have been proposed, these studies mostly focus on a single modality or a single scene, and there is still a lack of systematic exploration of the physical differences across modalities and the collaboration mechanism between backdoor and adversarial attacks, thereby exposing exploitable vulnerabilities. To this end, we propose physical multi-modal backdoor-adversarial example collaborative attack (PhysMCA) to achieve end-to-end design from digital domain optimization to physical domain verification. Our method includes two key innovations: (1) a multi-modal trigger localization algorithm for the chest-abdominal region based on human pose prior and Bayesian Optimization is proposed. Compared with traditional static template matching, the proposed method improves the accuracy, adaptability, and robustness of multi-modal target localization; (2) a joint optimization of backdoor implantation and adversarial perturbation in multi-modal models is proposed based on micro composite trigger and lightweight adversarial camouflage mechanism, which forms a multi-modal attack link with high concealment and poses significant challenges to existing single-attack detection mechanisms. The experimental results show that our method has excellent performance in both digital and physical domains. In the physical domain, the attack success rate in the visible light reaches 93.7%, and the attack success rate in the thermal infrared reaches 90.4%.

Primary Area: alignment, fairness, safety, privacy, and societal considerations

Submission Number: 9213