A Communication Efficient Federated Kernel $k$-Means

Xiaochen Zhou, Xudong Wang

Keywords: federated learning, kernel $k$-means, communication efficient

Abstract: A federated kernel $k$-means algorithm is developed in this paper. This algorithm resolves two challenging issues: 1) how to distributedly solve the optimization problem of kernel $k$-means under federated settings; 2) how to maintain communication efficiency in the algorithm. To tackle the first challenge, a distributed stochastic proximal gradient descent (DSPGD) algorithm is developed to determine an approximated solution to the optimization problem of kernel $k$-means. To tackle the second challenge, a communication efficient mechanism (CEM) is designed to reduce the communication cost. Besides, the federated kernel $k$-means provides two levels of privacy preservation: 1) users’ local data are not exposed to the cloud server; 2) the cloud server cannot recover users’ local data from the local computational results via matrix operations. Theoretical analysis shows: 1) DSPGD with CEM converges with an $O(1/T)$ rate, where $T$ is the number of iterations; 2) the communication cost of DSPGD with CEM is unrelated to the number of data samples; 3) the clustering quality of the federated kernel $k$-means approaches that of the standard kernel $k$-means, with a $(1+\epsilon)$ approximate ratio. The experimental results show that the federated kernel $k$-means achieves the highest clustering quality with the communication cost reduced by more than $60\%$ in most cases.

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics

Reviewed Version (pdf): https://openreview.net/references/pdf?id=IyB_e1BjGJ