Variational Inference in Similarity Spaces: A Bayesian Approach to Personalized Federated Learning

Pedro H Barros, Fabricio Murai, Amir Houmansadr, Alejandro C. Frery, Heitor Soares Ramos Filho

Keywords: Similarity spaces, Federated Learning, Variational Inference

Abstract: Similarity space (or S-space) employs an encoder function, fed by labeled original pairwise data, to find a latent pairwise space with markers (prototypical) vector. It divides the space into regions where pairs of objects are either similar or dissimilar. This paper enhances S-space, equipping variational inference from personalized federated learning. The S-space representation aligns local representation spaces across clients, while variational inference improves generalization and reduces overfitting caused by data scarcity and client heterogeneity. Our theoretical analysis improved upper bounds on KL divergence between optimal local and optimal global variational models compared to traditional distributed Bayesian neural networks.

Submission Number: 72