Learnable Invisible Backdoor for Diffusion Models

Sen Li, Junchi Ma, Minhao Cheng

Primary Area: societal considerations including fairness, safety, privacy

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Keywords: Backdoor attack, Diffusion models

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.

TL;DR: We propose a novel and general optimization framework to learn invisible backdoor that is applicable for both unconditional and conditional diffusion models.

Abstract: Diffusion models have shown tremendous potential for high-quality image generation in recent years. Accordingly, there has been a rising focus on security threats associated with diffusion models, primarily because of their potential for malicious utilization. Recent studies have shown diffusion models are vulnerable to backdoor attack, which can make diffusion models generate designated target images given corresponding triggers. However, current backdoor attacks depend on manually designed trigger generation functions, which are usually visible patterns added to input noise, making them easily detected by human inspection. In this paper, we propose a novel and general optimization framework to learn invisible trigger, making the inserted backdoor more stealthy and robust. Our proposed framework can be applied to both unconditional and conditional diffusion models. In addition, for conditional diffusion models, we are the first to show how to backdoor diffusion models in text-guided image editing/inpainting pipeline. Extensive experiments on various commonly used samplers and datasets verify the effectiveness and stealthiness of the proposed framework.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 7326