Greedy Poisson Rejection Sampling

Gergely Flamich

Keywords: channel simulation, relative entropy coding, reverse channel coding, rejection sampling, Poisson process

TL;DR: We propose rejection sampling procedures using Poisson processes, dual to A* sampling in a certain sense. Then, we use it to develop algorithms to optimally compress samples from any distribution, which is useful for neural data compression.

Abstract: One-shot channel simulation is a fundamental data compression problem concerned with encoding a single sample from a target distribution $Q$ using a coding distribution $P$ using as few bits as possible on average. Algorithms that solve this problem find applications in neural data compression and differential privacy and can serve as a more efficient and natural alternative to quantization-based methods. Unfortunately, existing solutions are too slow or have limited applicability, preventing their widespread adaptation. In this paper, we conclusively solve one-shot channel simulation for one-dimensional problems where the target-proposal density ratio is unimodal by describing an algorithm with optimal runtime. We achieve this by constructing a rejection sampling procedure equivalent to greedily searching over the points of a Poisson process. Hence, we call our algorithm greedy Poisson rejection sampling (GPRS) and analyze the correctness and time complexity of several of its variants. Finally, we empirically verify our theorems, demonstrating that GPRS significantly outperforms the current state-of-the-art method, A* coding.

Submission Number: 3292