Densely connected normalizing flowsDownload PDF

21 May 2021, 20:46 (edited 26 Oct 2021)NeurIPS 2021 PosterReaders: Everyone
  • Keywords: Density estimation, Normalizing flows, Image generation, DenseFlow, Densely connected couplings
  • TL;DR: Normalizing flows with densely connected couplings achieve state-of-the-art performance under moderate computation budgets.
  • Abstract: Normalizing flows are bijective mappings between inputs and latent representations with a fully factorized distribution. They are very attractive due to exact likelihood evaluation and efficient sampling. However, their effective capacity is often insufficient since the bijectivity constraint limits the model width. We address this issue by incrementally padding intermediate representations with noise. We precondition the noise in accordance with previous invertible units, which we describe as cross-unit coupling. Our invertible glow-like modules increase the model expressivity by fusing a densely connected block with Nyström self-attention. We refer to our architecture as DenseFlow since both cross-unit and intra-module couplings rely on dense connectivity. Experiments show significant improvements due to the proposed contributions and reveal state-of-the-art density estimation under moderate computing budgets.
  • Supplementary Material: pdf
  • Code Of Conduct: I certify that all co-authors of this work have read and commit to adhering to the NeurIPS Statement on Ethics, Fairness, Inclusivity, and Code of Conduct.
  • Code: https://github.com/matejgrcic/DenseFlow
13 Replies

Loading