Rethinking Memorization–Generalization Trade-Off in Generative Models

Jiseok Chae, Kyuwon Kim, Donghwan Kim

Keywords: Generative models, Generalization, Learning theory

TL;DR: We demonstrate that benign overfitting can occur in overparameterized generative models.

Abstract: Existing generative models exhibit a memorization–generalization trade-off, and thus, avoiding memorization is a common strategy to promote generalization. In supervised learning, this long-accepted trade-off is being challenged, as recent studies show modern overparametrized models can achieve benign overfitting; that is, they generalize well even while exactly fitting, or memorizing, the training data. This raises the question of whether overparameterized generative models can similarly bypass this trade-off and achieve superior generalization alongside memorization. We address this with an empirical risk formulation that uses presampled latent variables instead of integrating over the entire latent distribution. We then recast the generative modeling problem as a supervised learning task of learning an optimal transport map, enabling us to leverage the concept of benign overfitting. In the one-dimensional setting, we show for the first time that benign overfitting can occur in generative models. We further expand and empirically validate our approach to higher dimensions, illustrating that benign overfitting extends more broadly across generative models.

Student Paper: Yes

Submission Number: 90