Harnessing the Power of Federated Learning in Federated Contextual Bandits

Chengshuai Shi, Ruida Zhou, Kun Yang, Cong Shen

Keywords: Contextual Bandits, Federated Learning

TL;DR: This work addresses the current disconnection between the studies of federated learning (FL) and federated contextual bandits (FCB) by proposing an innovative FCB design, which is capable of harnessing the full spectrum of FL advances.

Abstract: Federated contextual bandits (FCB), a pivotal integration of federated learning (FL) and sequential decision-making, has garnered significant attention in recent years. Prior research on FCB can be understood as specific instantiations of a unified design principle articulated in this paper: "FCB = FL + CB". Here, FL enhances agents' performance by aggregating the information of other agents' local data to better contextual bandits (CB) policies. Nevertheless, it is evident that existing approaches largely employ tailored FL protocols, often deviating from the canonical FL framework. Consequently, even renowned algorithms like FedAvg remain underutilized in FCB, let alone other FL advancements. To bridge this gap between the canonical FL study and the FL component in FCB, our work introduces a novel FCB design, termed FedIGW, that incorporates inverse gap weighting as the CB algorithm. This design permits the integration of versatile FL protocols as long as they can solve a standard FL problem. With this flexible FL choice, FedIGW advances FCB research by enabling the utilization of the entire spectrum of FL innovations, encompassing canonical algorithmic designs (e.g., FedAvg and SCAFFOLD), convergence analyses, and valuable extensions (such as personalization, robustness, and privacy). We substantiate these claims through rigorous theoretical analyses and empirical evaluations.

Primary Area: reinforcement learning

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 4551