Computing Low-Entropy Couplings for Large-Support Distributions
Primary Area: general machine learning (i.e., none of the above)
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Keywords: minimum-entropy coupling
Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.
TL;DR: Algorithm for producing low-entropy couplings for distributions with large supports
Abstract: A minimum-entropy coupling is a joint probability distribution having minimum joint entropy among all joint distributions with given pre-specified marginals. While provable approximation algorithms for a minimum-entropy coupling exist, they take log-linear time in the size of the support of the marginal distributions. Thus, applications involving very large-support distributions instead use a class of heuristic iterative minimum-entropy coupling (IMEC) algorithms. Unfortunately, existing IMEC algorithms are limited to specific classes of distributions, prohibiting applications involving general large-support distributions. In this work, we resolve this issue by making three main contributions: 1) We unify existing IMEC algorithms under a single formalism using sets of partitions. 2) We derive a new IMEC instance from this formalism, which we call ARIMEC, that, unlike existing IMEC algorithms, can be applied to arbitrary discrete distributions; furthermore, we introduce associated operations that make ARIMEC efficient in practice. 3) We empirically illustrate the utility of ARIMEC for both Markov coding games and steganography.
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Submission Number: 4090
Loading