Overcoming Alignment Constraints: G-Patch for Practical Adversarial Attacks on ViTs

Mingzhen Shao

Keywords: computer vision, adversarial attack, vision transformer, practical deployment

TL;DR: The inaugural real-world deployable adversarial attack patch designed for vision transformers.

Abstract: This paper addresses the vulnerability of adversarial patches designed for vision transformers, which traditionally depended on precise alignment with patch locations. Such alignment constraints hindered practical deployment in the physical world. We propose the G-Patch, a novel method for generating adversarial patches that overcomes this constraint, enabling targeted attacks from any position within the field of view. Instead of directly optimizing the patch using gradients, we employ a sub-network structure for patch generation. Our experiments demonstrate the G-Patch's effectiveness in achieving universal attacks on vision transformers with a small size. Further analysis shows its resilience to challenges like brightness restriction, color transfer, and random noise, enhancing robustness and inconspicuousness in real-world deployments. Black box and real-world attack experiments validate its effectiveness even under challenging conditions.

Primary Area: general machine learning (i.e., none of the above)

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 1487