On a Relation Between the Rate-Distortion Function and Optimal Transport

Eric Lei, Hamed Hassani, Shirin Saeedi Bidokhti

Keywords: optimal transport, rate-distortion, information theory, Sinkhorn

TL;DR: We establish a connection between rate-distortion and optimal transport, and demonstrate how optimal transport solvers can be used to compute solutions to source coding problems.

Abstract: We discuss a relationship between rate-distortion and optimal transport (OT) theory, even though they seem to be unrelated at first glance. In particular, we show that a function defined via an extremal entropic OT distance is equivalent to the rate-distortion function. We numerically verify this result as well as previous results that connect the Monge and Kantorovich problems to optimal scalar quantization. Thus, we unify solving scalar quantization and rate-distortion functions in an alternative fashion by using their respective optimal transport solvers.