Large Sparse Kernels for Federated LearningDownload PDF

01 Mar 2023 (modified: 31 May 2023)Submitted to Tiny Papers @ ICLR 2023Readers: Everyone
Keywords: Federated Learning, Non-IID
TL;DR: We demonstrate that large sparse convolution kernels can improve robustness against various Non-IID problem in federated learning.
Abstract: Existing approaches to address non-iid data in federated learning are often tailored to specific types of heterogeneity and may lack generalizability to all scenarios. In this paper, we present empirical evidence supporting the claim that employing large sparse convolution kernels can lead to enhanced robustness against distribution shifts in the context of federated learning for various non-iid problems, including imbalanced data volumes, different feature spaces, and label distributions. Our experimental results demonstrate that the substitution of convolutional kernels with large sparse kernels can yield substantial improvements in the ability to resist non-iid problems across multiple methods.
7 Replies