Tackling heterogeneity in medical federated learning via aligning vision transformers

Erfan Darzi, Yiqing Shen, Yangming Ou, Nanna Maria Sijtsema, Peter M. A. van Ooijen

Abstract: Highlights•Vision Transformer-based federated learning improves accuracy in heterogeneous settings.•Multi-head attention alignment enhances fairness for underrepresented clients.•Weighted averaging boosts performance, especially in highly heterogeneous environments.•Vision Transformer approach reduces need for complex multi-level optimization in FL.