Looking Beyond Aggregation for Medical Federated Learning: From Analysis to Novel Architecture Design

Vasilis Siomos, Nils Hallerfelt, Jonathan Passerat-Palmbach, Giacomo Tarroni

Keywords: Medical Imaging, Federated Learning

Abstract:

Federated Learning (FL) offers a privacy-preserving pathway for collaborative model development
across medical institutions, making it particularly valuable for medical imaging applications where
data cannot be shared. However, the statistical heterogeneity of multi-center medical data, where
each institution’s images vary in acquisition protocols, patient populations, and disease prevalence,
severely challenges FL performance. Medical FL research has extensively focused on developing
improved aggregation algorithms to combat this heterogeneity, meaning two other aspects of the
pipeline, namely the initialization strategy and the model architecture, have remained an under-
explored frontier. For initialization, we know task-relevant pre-training through self-supervised
learning (SSL) is a highly-effective alternative to ImageNet (IN) pre-training, even more so in
the data-scarce and costly annotation medical landscape, but the potential of SSL in the FL setting
remains largely unexplored. In terms of architectures, it is common for FL papers to present novel
aggregation methods tested on shallow/toy networks, which do not mirror the deep and complex
architectures deployed in real-world applications. In this paper, we present a two-stage investigation
to fill this gap.

Submission Number: 35