Semi-Discrete Normalizing Flows through Differentiable Voronoi TessellationDownload PDF

Published: 29 Mar 2022, Last Modified: 05 May 2023ICLR 2022 DGM4HSD workshop OralReaders: Everyone
TL;DR: We combine differentiable tessellation with invertible transformations to convex polytopes to construct a new normalizing flow that has learnable discrete structure, and can map between discrete and continuous random variables.
Abstract: Mapping between discrete and continuous distributions is a difficult task and many have had to resort to approximate or heuristical approaches. We propose a tessellation-based approach that directly learns quantization boundaries on a continuous space, complete with exact likelihood evaluations. This is done through constructing normalizing flows on convex polytopes defined via a differentiable tessellation. Using a simple homeomorphism with an efficient log determinant Jacobian, we can then cheaply parameterize distributions on bounded domains. We explore this approach in two application settings, mapping from discrete to continuous and vice versa. Firstly, a Voronoi dequantization allows automatically learning quantization boundaries in a multidimensional space. The location of boundaries and distances between regions can encode useful structural relations between the quantized discrete values. Secondly, a Voronoi mixture model has constant computation cost for likelihood evaluation regardless of the number of mixture components. Empirically, we show improvements over existing methods across a range of structured data modalities.
3 Replies