On the advantages of stochastic encodersDownload PDF

Mar 04, 2021 (edited Apr 01, 2021)Neural Compression Workshop @ ICLR 2021Readers: Everyone
  • Keywords: compression, autoencoders, universal quantization, perception-distortion trade-off, realism
  • TL;DR: An example showing that stochastic encoders can outperform deterministic encoders when optimizing realism in addition to distortion
  • Abstract: Stochastic encoders have been used in rate-distortion theory and neural compression because they can be easier to handle. However, in performance comparisons with deterministic encoders they often do worse, suggesting that noise in the encoding process may generally be a bad idea. It is poorly understood if and when stochastic encoders can do better than deterministic encoders. In this paper we provide one illustrative example which shows that stochastic encoders can significantly outperform the best deterministic encoders. Our toy example suggests that stochastic encoders may be particularly useful in the regime of "perfect perceptual quality".
1 Reply