Large Sparse Kernels for Federated Learning

Feilong Zhang, Yinchuan Li, Shiyi Lin, yunfeng shao, Junjun Jiang, Xianming Liu

Keywords: Federated Learning, Non-IID

TL;DR: We demonstrate that large sparse convolution kernels can improve robustness against various Non-IID problem in federated learning.

Abstract: Existing approaches to address non-iid data in federated learning are often tailored to specific types of heterogeneity and may lack generalizability to all scenarios. In this paper, we present empirical evidence supporting the claim that employing large sparse convolution kernels can lead to enhanced robustness against distribution shifts in the context of federated learning for various non-iid problems, including imbalanced data volumes, different feature spaces, and label distributions. Our experimental results demonstrate that the substitution of convolutional kernels with large sparse kernels can yield substantial improvements in the ability to resist non-iid problems across multiple methods.