Sliced Iterative Normalizing Flows

Biwei Dai, Uros Seljak

Keywords: Generative Models, Normalizing Flow, Optimal Transport, Sliced Wasserstein Distance

TL;DR: An iterative sliced optimal transport based normalizing flow that is easy to train, fast and achieves competitive results on image generation and density estimation tasks.

Abstract: We develop an iterative (greedy) deep learning algorithm which is able to transform an arbitrary probability distribution function (PDF) into the target PDF. The model is based on iterative Optimal Transport of a series of 1D slices, matching on each slice the marginal PDF to the target. As special cases of this algorithm, we introduce two sliced iterative Normalizing Flow (SINF) models, which map from the data to the latent space (GIS) and vice versa (SIG). We show that SIG is able to generate high quality samples of image datasets, which match the GAN benchmarks. GIS obtains competitive results on density estimation tasks compared to the density trained NFs. SINF has very few hyperparameters and is very stable during training. When trained on small training sets it is both faster and achieves higher $p(x)$ than current alternatives.