TooBad: Backdoor Diffusion Models with Ultra-Low Poison Rate and Imperceptible Trigger

Vu Truong, Long Bao Le

Keywords: Diffusion models, backdoor attack, trigger optimization, ultra-low poison rate, imperceptible trigger

Abstract: Diffusion models (DMs), despite their impressive capabilities across a wide range of generative tasks, have been shown to be vulnerable to backdoor attacks. However, existing backdoor methods face critical trade-offs among key factors: attack performance, stealthiness, time complexity, and practicality (e.g., poison rate requirement). For example, achieving high attack performance typically demands a high poison rate and prolonged training, which undermines stealthiness, making the attack more detectable by backdoor defenses. Furthermore, high poison rates are often infeasible in real-world scenarios where attackers can poison only a minimal fraction of the training data. In this paper, we propose TooBad (trigger optimization for backdoor diffusion models), a novel attack framework which can implant backdoors into DMs with an extremely low poison rate, while achieving higher attack success rate (ASR), stronger resistance to backdoor defenses, and significantly reduced training time compared to existing backdoor attacks. Experiments show that TooBad can achieve approximately 82% ASR at a 0.5% poison rate, significantly lower than the 10% poison rates required by prior work. At 5% poison rate, TooBad reaches nearly 100% ASR within just 3-5 training epochs, whereas existing methods need at least 30-50 epochs at double the poison rate for comparable results. Despite its potency, TooBad easily evades state-of-the-art (SOTA) defenses while maintaining high utility (i.e., the capability of generating clean samples). These results reveal a critical threat on DMs and highlight the urgent need for more robust defenses against such stealthy yet efficient attacks.

Primary Area: generative models

Submission Number: 7579