Collaborative Best Arm Identification with Limited Communication on Non-IID Data

Nikolai Karpov, Qin Zhang

Abstract: In this paper, we study the tradeoffs between the time speedup and the round complexity in the collaborative learning model with non-IID data, where multiple agents interact with possibly different environments and they want to learn an objective in the aggregated environment. We use a basic problem in bandit theory called best arm identification in multi-armed bandits as a vehicle to deliver the following conceptual message: collaborative learning on non-IID data is provably more difficult than that on IID data. In particular, we show the following: 1) Learning time speedup in the non-IID data setting can be much smaller than $1$ (that is, a slowdown). When the number of rounds $R = O(1)$, we will need at least a polynomial number of agents (in terms of the number of arms) to achieve a speedup $\tilde{\Omega}(1)$. This is in stark contrast to the IID data setting, where the speedup is always $\tilde{\Omega}(1)$ regardless of $R$ and the number of agents $K$. 2) Local adaptivity of the agents cannot help much in the non-IID data setting. This is in contrast with the IID data setting, in which to achieve the same speedup, the best non-adaptive algorithm requires a significantly larger number of rounds than the best adaptive algorithm.