Understanding and Mitigating Memorization in Generative Models via Sharpness of Probability Landscapes

Dongjae Jeon, Dueun Kim, Albert No

TL;DR: We propose a sharpness-based framework to analyze and mitigate memorization in diffusion models through novel metrics and initial point adjustment strategies.

Abstract: In this paper, we introduce a geometric framework to analyze memorization in diffusion models through the sharpness of the log probability density. We mathematically justify a previously proposed score-difference-based memorization metric by demonstrating its effectiveness in quantifying sharpness. Additionally, we propose a novel memorization metric that captures sharpness at the initial stage of image generation in latent diffusion models, offering early insights into potential memorization. Leveraging this metric, we develop a mitigation strategy that optimizes the initial noise of the generation process using a sharpness-aware regularization term.

Lay Summary:

Generative models like Stable Diffusion can produce impressive images, but they sometimes memorize parts of their training data. This raises privacy concerns, especially when the training data contains sensitive content.

We study this memorization through a geometric perspective, showing that it emerges in regions where the model’s predictions are unusually sharp. By analyzing the Hessian of the log probability, we quantify this sharpness and demonstrate that it enables early detection of memorization during the generation process.

We also provide a theoretical explanation for Wen et al.'s popular detection metric, showing that it captures sharpness differences between conditional and unconditional predictions. Building on this insight, we develop an enhanced metric by amplifying these sharpness signals using the Hessian.

Based on our sharpness framework, we introduce SAIL, a method that reduces memorization by selecting smoother initial noise during sampling. SAIL operates entirely at inference time and requires no changes to the model or prompt.

Our work offers both theoretical insight and a practical tool for improving the safety of generative models.

Primary Area: Deep Learning->Generative Models and Autoencoders

Keywords: Diffusion Model, Memorization, Sharpness Aware

Submission Number: 15448