Secure and Efficient Watermarking for Latent Diffusion Models in Model Distribution Scenarios

Liangqi Lei, Keke Gai, Jing Yu, Liehuang Zhu, Qi Wu

Abstract:

Latent diffusion models have exhibited consider
able potential in generative tasks. Watermarking
is considered to be an alternative to safeguard the
copyright of generative models and prevent their
misuse. However, in the context of model distribu
tion scenarios, the accessibility of models to large
scale of model users brings new challenges to the
security, efficiency and robustness of existing wa
termark solutions. To address these issues, we pro
pose a secure and efficient watermarking solution.
A new security mechanism is designed to prevent
watermark leakage and watermark escape, which
considers watermark randomness and watermark
model association as two constraints for manda
tory watermark injection. To reduce the time cost
of training the security module, watermark injec
tion and the security mechanism are decoupled, en
suring that fine-tuning VAE only accomplishes the
security mechanism without the burden of learn
ing watermark patterns. A watermark distribution
based verification strategy is proposed to enhance
the robustness against diverse attacks in the model
distribution scenarios. Experimental results prove
that our watermarking consistently outperforms ex
isting six baselines on effectiveness and robust
ness against ten image processing attacks and ad
versarial attacks, while enhancing security in the
distribution scenarios. Our code is available at
https://github.com/REPO-EXP/DistriMark.