A Wasserstein Minimum Velocity Approach to Learning Unnormalized Models

Ziyu Wang, Shuyu Cheng, Yueru Li, Jun Zhu, Bo Zhang

Keywords: energy-based model, optimal transport, variational auto-encoder, Wasserstein auto-encoder, score matching

TL;DR: We present a scalable approximation to a wide range of EBM objectives, and applications in implicit VAEs and WAEs

Abstract: Score matching provides an effective approach to learning flexible unnormalized models, but its scalability is limited by the need to evaluate a second-order derivative. In this paper,we connect a general family of learning objectives including score matching to Wassersteingradient flows. This connection enables us to design a scalable approximation to theseobjectives, with a form similar to single-step contrastive divergence. We present applications in training implicit variational and Wasserstein auto-encoders with manifold-valued priors.

Community Implementations: [ 1 code implementation](https://www.catalyzex.com/paper/arxiv:2002.07501/code)