Adaptive Latent-Space Constraints in Personalized Federated Learning

Sana Ayromlou, D. B. Emerson

Keywords: Federated Learning, Maximum Mean Discrepancy, Data Heterogeneity, Deep Learning, Medical Imaging

TL;DR: This paper demonstrates that applying adaptive latent-space constraints in personalized FL algorithms improves performance across a number of challenging benchmark tasks, especially those with significant feature heterogeneity

Abstract: Federated learning (FL) is an effective and widely used approach to training deep learning models on decentralized datasets held by distinct clients. FL also strengthens both security and privacy protections for training data. Common challenges associated with statistical heterogeneity between distributed datasets have spurred significant interest in personalized FL (pFL) methods, where models combine aspects of global learning with local modeling specific to each client’s unique characteristics. This work investigates the efficacy of theoretically supported, adaptive MMD measures in pFL, primarily focusing on the Ditto framework, a state-of-the-art technique for distributed data heterogeneity. The use of such measures significantly improves model performance across a variety of tasks, especially those with pronounced feature heterogeneity. Additional experiments demonstrate that such measures are directly applicable to other pFL techniques and yield similar improvements across a number of datasets. Finally, the results motivate the use of constraints tailored to the various kinds of heterogeneity expected in FL systems.

Primary Area: Deep learning (e.g., architectures, generative models, optimization for deep networks, foundation models, LLMs)

Submission Number: 5157