Tackling Personalized Federated Learning with Label Concept Drift via Hierarchical Bayesian Modeling

Xingchen Ma, Junyi Zhu, Matthew B. Blaschko

Keywords: Label concept drift, federated learning, variational inference

TL;DR: We propose a general framework for Personalized Federated Learning with label concept drift based on the hierarchical Bayesian inference.

Abstract: Federated Learning (FL) is a distributed learning scheme to train a shared model across clients. One fundamental challenge in FL is that the sets of data across clients could be non-identically distributed. Personalized Federated Learning (PFL) attempts to solve this challenge. Most methods in the literature of PFL focus on the data heterogeneity that clients differ in their label distributions. In this work, we focus on label concept drift which is a broad but relatively unexplored area. We present a general framework for PFL based on hierarchical Bayesian inference and propose a variational inference algorithm based on this framework. We demonstrate our methods through empirical studies on CIFAR100 and SUN397. Experimental results show our approach significantly outperforms the baselines when tackling the label concept drift across clients.

Is Student: Yes