On the Diminishing Reliability of Reference-Free Memorization Detection in Modern Diffusion Models

Anjun Hu, Philip Torr, Francesco Pinto

Keywords: Diffusion Models, Training Data Memorization, Model Safety, Multi-Modal Training, 3D Generation, Video Generation, Machine Unlearning

TL;DR: Memorization detection metrics that work well on standard text-to-image diffusion models suffer from efficacy degradation on DMs with more varied training protocols.

Abstract: Diffusion models have been observed to memorize and regurgitate portions of their training data, which raises potential copyright and privacy concerns. To quantify and mitigate this phenomenon, various reference-free metrics that operate without training data access have become an effective tool for detecting memorization in text-to-image systems. As diffusion models expand beyond the familiar text-to-image paradigm to encompass multi-modal and multi-stage training for 3D and video synthesis, the reliability of existing detection methods in these novel domains remains unclear. In this work, (1) We find that metric efficacy declines when applied to models that are fine-tuned in multiple stages from a text-to-image base to support additional modalities, where more varied training protocols may obscure memorization signals from existing detection techniques. (2) We demonstrate that these metrics have limited reliability in distinguishing between successful and failed memorization mitigation attempts, risking false judgments in model sanitization efforts. (3) We trace this performance degradation to violations of assumptions underlying current detection frameworks and conduct factorized analysis. Our findings call for caution when applying existing memorization detection metrics beyond text-to-image models and point toward the need for more robust evaluation methods tailored to a wider range of emerging diffusion models with diverse training protocols.

Primary Area: alignment, fairness, safety, privacy, and societal considerations

Submission Number: 1722