Discrete Denoising FlowsDownload PDF

Published: 15 Jun 2021, Last Modified: 05 May 2023INNF+ 2021 spotlighttalkReaders: Everyone
Keywords: Discrete Flows, Lossless Compression
TL;DR: We introduce a new discrete flow-based generative model for categorical random variables, called Discrete Denoising Flows (DDFs).
Abstract: Discrete flow-based models are a recently proposed class of generative models that learn invertible transformations for discrete random variables. Since they do not require data dequantization and maximize an exact likelihood objective, they can be used in a straight-forward manner for lossless compression. In this paper, we introduce a new discrete flow-based model for categorical random variables: Discrete Denoising Flows (DDFs). In contrast with other discrete flow-based models, our model can be locally trained without introducing gradient bias. We show that DDFs outperform Discrete Flows on modelling a toy example, binary MNIST and Cityscapes segmentation maps, measured in log-likelihood.
4 Replies