Importance weighted compressionDownload PDF

04 Mar 2021, 19:45 (edited 01 Apr 2021)Neural Compression Workshop @ ICLR 2021Readers: Everyone
  • Keywords: compression, variational autoencoder, importance weighted autoencoder, bits-back coding, softmin coding, gumbel
  • TL;DR: The loss of an IWAE can be interpreted as the coding cost of a lossless compression scheme or the rate-distortion trade-off of a lossy compression scheme
  • Abstract: The connection between variational autoencoders (VAEs) and compression is well established and they have been used for both lossless and lossy compression. Compared to VAEs, importance-weighted autoencoders (IWAEs) achieve a larger bound on the log-likelihood. However, it is not well understood whether a similar connection between IWAEs and compression exists and whether the improved loss corresponds to better compression performance. Here we show that the loss of IWAEs can indeed be interpreted as the cost of lossless or lossy compression schemes, and using IWAEs for compression can lead to small improvements in performance.
1 Reply