FedDAG: Clustered Federated Learning via Global Data and Gradient Integration for Heterogeneous Environments

ICLR 2026 Conference Submission15820 Authors

Keywords: Federated Learning, Clustering, Distributed Machine Learning

TL;DR: We introduce FEDDAG, a clustered FL approach that tackles data heterogeneity by combining data and gradient similarity for improved client clustering, and employs a dual-encoder architecture to enable representation sharing across clusters.

Abstract:

Federated Learning (FL) enables a group of clients to collaboratively train a model without sharing individual data, but its performance drops when client data are heterogeneous. Clustered FL tackles this by grouping similar clients. However, existing clustered FL approaches rely solely on either data similarity or gradient similarity; however, this results in an incomplete assessment of client similarities. Prior clustered FL approaches also restrict knowledge and representation sharing to clients within the same cluster. This prevents cluster models from benefiting from the diverse client population across clusters.
To address these limitations, FEDDAG introduces a clustered FL framework, FEDDAG, that employs a weighted, class-wise similarity metric that integrates both data and gradient information, providing a more holistic measure of similarity during clustering. In addition, FEDDAG adopts a dual-encoder architecture for cluster models, comprising a primary encoder trained on its own clients' data and a secondary encoder refined using gradients from complementary clusters. This enables cross-cluster feature transfer while preserving cluster-specific specialization.
Experiments on diverse benchmarks and data heterogeneity settings show that FEDDAG consistently outperforms state-of-the-art clustered FL baselines in accuracy.

Primary Area: other topics in machine learning (i.e., none of the above)

Submission Number: 15820