Interaction-Aware Gaussian Weighting for Clustered Federated Learning

Alessandro Licciardi, Davide Leo, Eros Fanì, Barbara Caputo, Marco Ciccone

TL;DR: We propose FedGWC a new clustering algorithm for Federated Learning and a new clustering metric tailored for heterogeneous FL

Abstract:

Federated Learning (FL) emerged as a decentralized paradigm to train models while preserving privacy. However, conventional FL struggles with data heterogeneity and class imbalance, which degrade model performance.
Clustered FL balances personalization and decentralized training by grouping clients with analogous data distributions, enabling improved accuracy while adhering to privacy constraints. This approach effectively mitigates the adverse impact of heterogeneity in FL.
In this work, we propose a novel clustering method for FL, **FedGWC** (Federated Gaussian Weighting Clustering), which groups clients based on their data distribution, allowing training of a more robust and personalized model on the identified clusters. **FedGWC** identifies homogeneous clusters by transforming individual empirical losses to model client interactions with a Gaussian reward mechanism. Additionally, we introduce the *Wasserstein Adjusted Score*, a new clustering metric for FL to evaluate cluster cohesion with respect to the individual class distribution. Our experiments on benchmark datasets show that **FedGWC** outperforms existing FL algorithms in cluster quality and classification accuracy, validating the efficacy of our approach.

Lay Summary:

Training AI models usually requires centralizing vast amounts of data, which raises privacy concerns. Federated Learning (FL) offers a solution by allowing edge devices or institutions - such as smartphones and hospitals - to train a shared model collaboratively without sending their private data to a central server. However, real-world data is often messy: different devices might have very diverse types of data, or some data categories might be rare on some devices whilst common on others. This *data heterogeneity* makes it hard for FL models to perform well across all devices.

Our work introduces **FedGWC**, a new method to make FL training more effective. Instead of forcing all devices to train one model, FedGWC groups devices with similar data characteristics into clusters, allowing each cluster to train its specialized model, which is much better suited to the data within that group. Think of it like organizing a study group: instead of everyone studying the same broad topic, smaller groups form to focus on specific subjects they all need help with. FedGWC does this by analyzing how well each device’s model learns from its own data without actually looking at the data itself. We also developed a new way to measure how good these clusters are, especially when some data categories are much rarer than others.

Our experiments show that FedGWC significantly improves the accuracy of models in FL setups, especially when data is diverse and unevenly distributed. This means we can build more powerful and personalized AI applications while preserving sensitive private information.

Application-Driven Machine Learning: This submission is on Application-Driven Machine Learning.

Link To Code: https: //github.com/davedleo/FedGWC

Primary Area: General Machine Learning->Everything Else

Keywords: Federated Learning, Clustered Federated Learning

Submission Number: 6950