Unbiased Quantization of the $L_1$ Ball for Communication-Efficient Distributed Mean Estimation

Nithish Suresh Babu, Ritesh Kumar, Shashank Vatedka

TL;DR: We design a computationally efficient unbiased quantizer for the $L_1$ ball for communication-efficient distributed mean estimation and federated learning.

Abstract: We study the problem of unbiased minimum mean squared error quantization of the $L_1$ ball, with applications to distributed mean estimation and federated learning. Inspired by quantization of probability distributions using types, we design a novel computationally efficient unbiased quantization scheme for vectors that lie within the $L_1$ ball. We also derive upper bounds on the worst-case mean squared error achieved by our scheme and show that this is order optimal. We then use this to design polynomial (in the dimension of the input vectors)-time schemes for communication-efficient distributed mean estimation and distributed/federated learning, and demonstrate its effectiveness using simulations.

Submission Number: 451