FedBKT: Federated Learning with Model Heterogeneity via Bidirectional Knowledge Transfer with Mediator Model

Qi Min, Hui Wang, Peng Jia, Yuan Qiu, Chunhua Gu, Weichao Ding

Abstract: Federated learning (FL) enables the training of global models across devices without the need to access local data. However, the heterogeneity of models and data across devices poses significant challenges in aggregating and aligning global information. In existing FL methods, knowledge transfer between the server and clients mainly relies on soft labels, a method that is affected by heterogeneity and raises privacy and data collection concerns due to its dependence on additional datasets. This paper proposes a FL paradigm based on Bidirectional Knowledge Transfer, named FedBKT, which leverages a mediator model to collect global information from each client. To achieve this, we introduce a Knowledge extraction module. After aggregation on the server, knowledge is transferred from the mediator model to local models via the Knowledge Sharing module, facilitating efficient information extraction and sharing. Experimental results in heterogeneous scenarios show that FedBKT outperforms existing methods across multiple metrics and effectively mitigates the negative impact of heterogeneity on performance.

External IDs:dblp:conf/icic/MinWJQGD25