Mediate: Mixture Domain Model-Agnostic Federated Learning

Chang Liu, Yuwen Yang, Xun Cai, Yue Ding, Hongtao Lu

Abstract: Federated learning (FL) is a widely studied distributed machine learning paradigm that enables participants to conduct model training while preserving privacy. However, real-world FL scenarios face three major challenges: mixed domains, heterogeneous models, and non-i.i.d. label distribution. Existing FL methods cannot simultaneously handle all three constraints and often require a reduction in privacy protection levels (e.g., sharing encrypted feature information, model architecture, or label distribution statistics of participants), limiting the applicability of FL. In this work, we first propose a challenging “high-level heterogeneous ” scenario, i.e., an FL scenario where the feature space, model architecture, and label distribution are all heterogeneous. We then design an FL algorithm based on parameter decoupling and data-free knowledge distillation strategies to address this scenario without exposing any client’s private information. Experimentally, in the “high-level heterogeneous ” scenario using image and tabular data, our proposed algorithm outperforms other state-of-the-art (SOTA) algorithms.