What You Predict Shapes How You Memorize: Target-Parameterization and Memorization Dynamics in Flow Matching

Mohammed Al-Jaff, Amir Mehrpanah, Gustav Eje Henter, Hossein Azizpour, Danica Kragic

Keywords: flow matching, memorization, target parameterization, generalization, diffusion models, generative models

TL;DR: The choice of prediction target (x, ε, v) in flow matching changes when and how much models memorize, revealing parameterization as a previously unrecognized axis of memorization control.

Abstract: Flow matching models trained on finite data can eventually memorize parts or all of their training set: the finite-data optimum of the regression objective reproduces the empirical training distribution rather than the unknown data distribution. In practice, models often avoid this memorizing solution because of finite capacity, finite training time, architectural bias, and implicit regularization. Separately, work on target parameterization has shown that predicting clean data $x$, noise $\epsilon$, or velocity $v$ can change empirical behavior and sample quality; we ask whether it also changes when and how flow matching models memorize. We study this in controlled regimes where memorization is observable, sweeping training-set size and model capacity while holding architecture, optimizer, training budget, sampler, and evaluation protocol fixed, and measuring checkpoint-level FID, nearest-neighbor memorization, and memorization at best FID. Across dataset sizes and model capacities, we find a consistent ordering: the three parameterizations memorize at different rates and to different degrees, with the ordering persisting even at matched sample quality, and the prediction target that is best for sample quality is not necessarily the one that memorizes least. Beyond its known role as a lever on generation quality, target parameterization also shapes when and how memorization emerges during training, a connection that, to our knowledge, has not been previously established.

Submission Number: 209