On the regularization of Wasserstein GANs

Henning Petzka, Asja Fischer, Denis Lukovnikov

Abstract: Since their invention, generative adversarial networks (GANs) have become a popular approach for learning to model a distribution of real (unlabeled) data. Convergence problems during training are overcome by Wasserstein GANs which minimize the distance between the model and the empirical distribution in terms of a different metric, but thereby introduce a Lipschitz constraint into the optimization problem. A simple way to enforce the Lipschitz constraint on the class of functions, which can be modeled by the neural network, is weight clipping. Augmenting the loss by a regularization term that penalizes the deviation of the gradient norm of the critic (as a function of the network's input) from one, was proposed as an alternative that improves training. We present theoretical arguments why using a weaker regularization term enforcing the Lipschitz constraint is preferable. These arguments are supported by experimental results on several data sets.

TL;DR: A new regularization term can improve your training of wasserstein gans

Code: [ lukovnikov/improved_wgan_training](https://github.com/lukovnikov/improved_wgan_training) + [ 1 community implementation](https://paperswithcode.com/paper/?openreview=B1hYRMbCW)

Community Implementations: [ 2 code implementations](https://www.catalyzex.com/paper/on-the-regularization-of-wasserstein-gans/code)