Integer Networks for Data Compression with Latent-Variable Models

Johannes Ballé, Nick Johnston, David Minnen

Abstract: We consider the problem of using variational latent-variable models for data compression. For such models to produce a compressed binary sequence, which is the universal data representation in a digital world, the latent representation needs to be subjected to entropy coding. Range coding as an entropy coding technique is optimal, but it can fail catastrophically if the computation of the prior differs even slightly between the sending and the receiving side. Unfortunately, this is a common scenario when floating point math is used and the sender and receiver operate on different hardware or software platforms, as numerical round-off is often platform dependent. We propose using integer networks as a universal solution to this problem, and demonstrate that they enable reliable cross-platform encoding and decoding of images using variational models.

Keywords: data compression, variational models, network quantization

TL;DR: We train variational models with quantized networks for computational determinism. This enables using them for cross-platform data compression.