GradCFG: Gradient Inversion of Classifier-Free Guidance Diffusion Models

ICLR 2026 Conference Submission18254 Authors

Keywords: Gradient Inversion Attack, Diffusion Models, Classifier-Free Guidance, AI Safety, Dataset Reconstruction

Abstract: Gradient inversion attacks, as a means of privacy theft, have been extensively studied and applied in classifier models, yet research on gradient inversion for diffusion models, particularly classifier-free guidance (CFG) diffusion models, remains relatively underdeveloped. CFG models such as Stable Diffusion present significant challenges for such attacks due to their complex training mechanisms, including the high-dimensional search space caused by multimodal variables, the non-uniqueness of the noise $\epsilon$ solution space, and the difficulty in optimizing discrete time steps $t$. To address these challenges, this paper proposes a novel joint inversion framework featuring two core algorithmic innovations: the **GradCFG** algorithm, which integrates a four-variable co-optimization mechanism for simultaneous reconstruction of image latent variables $\mathbf{x}_0$, text embeddings $C_0$, noise $\epsilon$, and reparameterized continuous time steps $t$, alongside a periodic restart strategy for $\epsilon$ to enhance solution stability and generalization; and the **Inv-Sam** algorithm, a model-difference-based generation optimization method that leverages the generative capability disparities between pre-fine-tuning and post-fine-tuning models to restore high-resolution details through a reverse-forward diffusion editing process. Systematic experiments in CFG model fine-tuning scenarios demonstrate that the proposed method effectively achieves high-quality image-text joint reconstruction for various textual conditions ranging from concise descriptions to complex semantic combinations.

Primary Area: other topics in machine learning (i.e., none of the above)

Submission Number: 18254