Continuous Relaxation For The Multivariate Noncentral Hypergeometric DistributionDownload PDF

Published: 25 Mar 2022, Last Modified: 05 May 2023ICLR2022 OSC PosterReaders: Everyone
Keywords: hypergeometric, reparameterization, continuous relaxation
TL;DR: We present a continuous relaxation for the multivariate non-central hypergeometric distribution, which enables its integration into automatic differentiation frameworks
Abstract: Partitioning a set of elements into a given number of groups of a priori unknown sizes is an essential task in many applications. Due to hard constraints, it is a non-differentiable problem that prohibits its direct use in modern machine learning frameworks. Hence, previous works mostly fall back on suboptimal heuristics or simplified assumptions. The multivariate hypergeometric distribution offers a probabilistic formulation of sampling a given number of elements from multiple groups. Unfortunately, as a discrete probability distribution, it neither is differentiable. We propose a continuous relaxation for the multivariate noncentral hypergeometric distribution. We introduce an efficient and numerically stable sampling procedure that enables reparameterized gradients for the hypergeometric distribution and its integration into automatic differentiation frameworks. We additionally highlight its advantages on a weakly-supervised learning task.
3 Replies