Distributed Estimation with Multiple Samples per User: Sharp Rates and Phase Transition

Jayadev Acharya, Clement Louis Canonne, Yuhan Liu, Ziteng Sun, Himanshu Tyagi

Keywords: density estimation, distributed algorithms, communication constraints, probability distributions, statistical rates

TL;DR: We provide tight upper and lower bounds for distributed estimation of discrete probability distributions as a function of all parameters, when each of n users holds m samples but can only send L bits.

Abstract: We obtain tight minimax rates for the problem of distributed estimation of discrete distributions under communication constraints, where $n$ users observing $m $ samples each can broadcast only $\ell$ bits. Our main result is a tight characterization (up to logarithmic factors) of the error rate as a function of $m$, $\ell$, the domain size, and the number of users under most regimes of interest. While previous work focused on the setting where each user only holds one sample, we show that as $m$ grows the $\ell_1$ error rate gets reduced by a factor of $\sqrt{m}$ for small $m$. However, for large $m$ we observe an interesting phase transition: the dependence of the error rate on the communication constraint $\ell$ changes from $1/\sqrt{2^{\ell}}$ to $1/\sqrt{\ell}$.

Code Of Conduct: I certify that all co-authors of this work have read and commit to adhering to the NeurIPS Statement on Ethics, Fairness, Inclusivity, and Code of Conduct.