Position-color jointly optimized adversarial patch for attacking cross-modal visual-infrared dense prediction tasks

He Li, Wenyue He, Weihang Kong, Xingchen Zhang

Keywords: Adversarial Patch; Cross-Modal Attack; Joint Optimization

TL;DR: Explore the patch attack optimization on the challenging dense prediction tasks

Abstract: Currently, studies on adversarial patches in dense prediction tasks have predominantly focused on the visible modality, with significant limitations in both patch content and location optimization. Existing methods for position optimization rely on model outputs and limited applicability to diverse scenarios. Additionally,color optimization does not adapt to the specific scene characteristics,leading to insufficient overall applicability and practicality. To explore the potential security risks of visual-infrared multi-modal systems,this study proposes a position-color joint optimization method based on the global search mechanism for generating cross-modal adversarial patches. This method designs a single patch to achieve simultaneous attacks on both visible and infrared modalities. A fitness function constructed from model outputs is used to iteratively optimize the patch's position and color. During the optimization process, the patch's position and color are finely adjusted to enhance the attack's effectiveness. Meanwhile,via fine-grained learning of color features,the adversarial patch achieves adaptive color alignment with the current scene context,thus achieving a balance between attack performance and stealth. Experimental results fully validate the effectiveness of multi-modal adversarial patch attacks,providing new insights and methods for the security evaluation of visual-infrared systems.

Primary Area: alignment, fairness, safety, privacy, and societal considerations

Code Of Ethics: true

Submission Guidelines: true

Anonymous Url: true

No Acknowledgement Section: true

Submission Number: 24310