Distributed Learning of Unknown Games for HetNet Selection

Zhiming Huang, Jianping Pan

Abstract: Heterogeneous network (HetNet) selection is a challenging problem for wireless devices equipped with different radio access technologies (e.g., LTE, 5G, and WiFi), as clients in the same network often behave selfishly to compete for the common resources to maximize their own rewards (e.g., throughput). Most existing works often model the HetNet selection problem as a non-cooperative game among clients and find strategies to achieve equilibria. However, those works often assume a full-information setting where the number, actions, or rewards of other clients are known a priori. In practice, clients may have limited knowledge, i.e., they only observe their own rewards in the networks they currently attach to. To address the HetNet selection problem in the limited information setting, we model the problem as an unknown game repeated for an unknown number of rounds, and propose a distributed learning algorithm called Lights to achieve correlated equilibria in a polynomial number of rounds with provable bounds. Theoretically, we present a novel concentration bound for the reward estimator used in Lights based on a martingale analysis, which is the key to proving the convergence property of Lights. Furthermore, extensive experiments are conducted to verify the performance of the proposed Lights algorithm.