Generative model based on minimizing exact empirical Wasserstein distance

Akihiro Iohara, Takahito Ogawa, Toshiyuki Tanaka

Abstract: Generative Adversarial Networks (GANs) are a very powerful framework for generative modeling. However, they are often hard to train, and learning of GANs often becomes unstable. Wasserstein GAN (WGAN) is a promising framework to deal with the instability problem as it has a good convergence property. One drawback of the WGAN is that it evaluates the Wasserstein distance in the dual domain, which requires some approximation, so that it may fail to optimize the true Wasserstein distance. In this paper, we propose evaluating the exact empirical optimal transport cost efficiently in the primal domain and performing gradient descent with respect to its derivative to train the generator network. Experiments on the MNIST dataset show that our method is significantly stable to converge, and achieves the lowest Wasserstein distance among the WGAN variants at the cost of some sharpness of generated images. Experiments on the 8-Gaussian toy dataset show that better gradients for the generator are obtained in our method. In addition, the proposed method enables more flexible generative modeling than WGAN.

Keywords: Generative modeling, Generative Adversarial Networks (GANs), Wasserstein GAN, Optimal transport

TL;DR: We have proposed a flexible generative model that learns stably by directly minimizing exact empirical Wasserstein distance.