An Optimal Diffusion Approach to Quadratic Rate-Distortion Problems: New Solution and Approximation Methods

ICLR 2026 Conference Submission18437 Authors

Keywords: information theory, rate-distortion, diffusion processes, stochastic control

TL;DR: We exploit a connection between rate-distortion (RD) and optimal transport to estimate RD functions using diffusion processes.

Abstract:

When compressing continuous data, it is inevitable to suffer some loss of information, which occurs in some distortion at reconstruction. The Rate-Distortion function is the lowest rate possible for a code whose decoding allows a given amount of such distortion.
We exploit the connection between rate-distortion (RD) and entropic optimal transport to propose a novel stochastic control formulation, and use a classic result dating back to Schrödinger to show that the tradeoff between rate and mean squared error distortion is equivalent to a tradeoff between control energy and the differential entropy of the terminal state, whose probability law yields the reconstruction distribution. For a special class of sources, we show that the optimal control law and trajectory in the space of probability measures are given by solving a backward heat equation. In the more general case, our approach gives rise to a numerical solution method, estimating the RD function using diffusion processes with a constant diffusion coefficient. We demonstrate our method in various examples.

Primary Area: probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)

Submission Number: 18437