Prompting the Unseen: Detecting Hidden Backdoors in Black-Box Models

HUANG ZIXUAN, JIA-WEI CHEN, Zhipeng Zhang, Chia-Mu Yu

Keywords: visual prompting, model reprogramming, backdoor detection, poisoning

TL;DR: We propose a black-box model-level backdoor detection method, BProm.

Abstract: Visual prompting (VP) is a new technique that adapts well-trained frozen models for source domain tasks to target domain tasks. This study examines VP's benefits for black-box model-level backdoor detection. The visual prompt in VP maps class subspaces between source and target domains. We identify a misalignment, termed class subspace inconsistency, between clean and poisoned datasets. Based on this, we introduce BProm, a black-box model-level detection method to identify backdoors in suspicious models, if any. BProm leverages the low classification accuracy of prompted models when backdoors are present. Extensive experiments confirm BProm's effectiveness.

Primary Area: alignment, fairness, safety, privacy, and societal considerations

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.

Reciprocal Reviewing: I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 5531