Continuous Relaxation For The Multivariate Noncentral Hypergeometric Distribution

Thomas M. Sutter, Laura Manduchi, Alain Ryser, Julia E Vogt

Keywords: hypergeometric, reparameterization, continuous relaxation

TL;DR: We present a continuous relaxation for the multivariate non-central hypergeometric distribution, which enables its integration into automatic differentiation frameworks

Abstract: Partitioning a set of elements into a given number of groups of a priori unknown sizes is an essential task in many applications. Due to hard constraints, it is a non-differentiable problem that prohibits its direct use in modern machine learning frameworks. Hence, previous works mostly fall back on suboptimal heuristics or simplified assumptions. The multivariate hypergeometric distribution offers a probabilistic formulation of sampling a given number of elements from multiple groups. Unfortunately, as a discrete probability distribution, it neither is differentiable. We propose a continuous relaxation for the multivariate noncentral hypergeometric distribution. We introduce an efficient and numerically stable sampling procedure that enables reparameterized gradients for the hypergeometric distribution and its integration into automatic differentiation frameworks. We additionally highlight its advantages on a weakly-supervised learning task.