Data Cartography for Detecting Memorization Hotspots and Guiding Data Interventions in Generative Models

Laksh Patel, Neel N Shanbhag

Keywords: Generative Models, Data Cartography, Memorization Detection, Privacy Preservation, Uniform Stability, Influence Functions, Data-Centric Interventions, Forget Events

TL;DR: GenDataCarto uses per‐sample difficulty and “forget”‐event scores to identify and down‐weight over-memorized examples, improving both privacy and generalization in generative model training.

Abstract:

Modern generative models risk overfitting and
unintentionally memorizing rare training exam-
ples, which can be extracted by adversaries or
inflate benchmark performance. We propose Gen-
erative Data Cartography (GenDataCarto), a
data-centric framework that assigns each pretrain-
ing sample a difficulty score (early-epoch loss)
and a memorization score (frequency of “forget
events”), then partitions examples into four quad-
rants to guide targeted pruning and up-/down-
weighting. We prove that our memorization score
lower-bounds classical influence under smooth-
ness assumptions and that down-weighting high-
memorization hotspots provably decreases the
generalization gap via uniform stability bounds.
Empirically, GenDataCarto reduces synthetic ca-
nary extraction success by over 40% at just 10%
data pruning, while increasing validation perplex-
ity by less than 0.5%. These results demonstrate
that principled data interventions can dramatically
mitigate leakage with minimal cost to generative
performance.

Submission Number: 4