Refinement Methods for Distributed Distribution Estimation under $\ell^p$-Losses

Deheng Yuan, Tao Guo, Zhongyi Huang

Keywords: Distributed estimation, distribution learning, communication constraints, distributed algorithms, optimal rate of convergence.

Abstract: Consider the communication-constrained estimation of discrete distributions under $\ell^p$ losses, where each distributed terminal holds multiple independent samples and uses limited number of bits to describe the samples. We obtain the minimax optimal rates of the problem for most parameter regimes. As a result, an elbow effect of the optimal rates at $p=2$ is clearly identified. In order to achieve the optimal rates for different parameter regimes, we introduce refinement methods and develop additional customized techniques in the estimation protocols. The general idea of the refinement methods is to first generate rough estimate by partial information and then establish refined estimate in subsequent steps guided by the rough estimate. Then customized techniques such as successive refinement, sample compression, thresholding and random hashing are leveraged to achieve the optimal rates in different parameter regimes. The optimality of the estimation protocols is shown by deriving compatible minimax lower bounds.

Primary Area: Theory (e.g., control theory, learning theory, algorithmic game theory)

Submission Number: 7162