AdPO: Enhancing the Adversarial Robustness of Large Vision-Language Models with Preference Optimization
Keywords: Adversarial defense, Adversarial attacks, Large Vision-Language Models, Preference optimization
TL;DR: An adversarial defense strategy for LVLMs based on preference optimization.
Abstract: Large Vision-Language Models (LVLMs), such as GPT-4 and LLaVA, have recently witnessed remarkable advancements and are increasingly being deployed in real-world applications.
However, inheriting the sensitivity of visual neural networks, LVLMs remain vulnerable to adversarial attacks, which can result in erroneous or malicious outputs.
While existing efforts utilize adversarial fine-tuning to enhance robustness, they often suffer from performance degradation on clean inputs.
In this paper, we proposes AdPO, a novel adversarial defense strategy for LVLMs based on preference optimization.
Preference optimization methods, such as DPO and RLHF, have been widely used to align large language models (LLMs) with human values and preferences.
For the first time, we reframe adversarial training as a preference optimization problem, aiming to enhance the model’s preference for generating normal outputs on clean inputs while rejecting the potential misleading outputs for adversarial examples.
Notably, AdPO achieves this by solely modifying the image encoder, e.g., CLIP ViT, resulting in superior robustness across a range of downstream tasks (including LVLMs and zero-shot classification).
Our comprehensive experimental validation confirms the efficacy of the proposed AdPO, which outperforms prior state-of-the-art methods.
Primary Area: alignment, fairness, safety, privacy, and societal considerations
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
Reciprocal Reviewing: I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Submission Number: 6088
Loading