Safe-SD: Safe and Traceable Stable Diffusion with Text Prompt Trigger for Invisible Generative Watermarking

Zhiyuan Ma, Guoli Jia, Biqing Qi, Bowen Zhou

Abstract: Recently, stable diffusion (SD) models have typically flourished in the field of image synthesis and personalized editing, with a range of photorealistic and unprecedented images being successfully generated. As a result, widespread interests have been ignited to develop and use various SD-based tools for visual content creations. However, the exposures of AI-created contents on public platforms could raise both legal and ethical risks. In this regard, the traditional methods of adding watermarks to the already generated images (i.e. post-processing) may face a dilemma (e.g., being erased or modified) in terms of copyright protection and content monitoring, since the powerful image inversion and text-to-image editing techniques have been widely explored in SD-based methods. In this work, we propose a $\textbf{Safe}$ and high-traceable $\textbf{S}$table $\textbf{D}$iffusion framework (namely $\textbf{Safe-SD}$) to adaptively implant the graphical watermarks (e.g., QR code) into the imperceptible structure-related pixels during generative diffusion process for supporting text-driven invisible watermarking and detection. Different previous high-cost injection-then-detection training framework, we design a simple and unified architecture, which makes it possible to simultaneously train watermark injection and detection in a single network, greatly improving the efficiency and convenience of use. Moreover, to further support text-driven generative watermarking and deeply explore its robustness and high-traceability, we elaborately design a $\lambda$-sampling and $\lambda$-encryption algorithm to fine-tune a latent diffuser wrapped by a VAE for balancing high-fidelity image synthesis and high-traceable watermark detection. We present our quantitative and qualitative results on two representative datasets LSUN, COCO and FFHQ, demonstrating state-of-the-art performance of Safe-SD and showing it significantly outperforms the previous approaches.

Primary Subject Area: [Generation] Social Aspects of Generative AI

Secondary Subject Area: [Generation] Generative Multimedia

Relevance To Conference: Thanks for this new setting that was carefully thought out for the multimedia community. We would like to state that the contribution of this paper is closely related to the multimedia or multimodal processing. Specifically, our safe and high-traceable generative watermarking scheme (i.e., Safe-SD) supports accepting multimedia information: text and images as input, and uses a text prompt trigger (also supports accepting multi-modal conditions such as text instruction, image canny, etc.) as a driver of watermark injection to adaptively implement invisible generative watermark injection for protecting the copyright of generated visual content such as images or videos. Moreover, it can be applied to numerous multimodal tasks such as: text-to-image generation, text-to-image editing, text-to-video (temporal frames), etc., for efficient content monitoring and copyright tracing in multimedia community, NOT just focusing on single-modal watermark injection and detection.

Submission Number: 3862