Robust Spike-based Decoupled Federated Information Bottleneck Learning with Spiking Neural Network under System Heterogeneity

Shuangming Yang, Junran Pan, Badong Chen

Keywords: Spiking neural network, federated learning, spike-based learning, neuromorphic computing

Abstract: As embedded devices become increasingly prevalent in intelligent systems, low-power system in resource-constrained environments has emerged as a key challenge. Spiking neural networks (SNNs), with their sparse and event-driven computation, have shown great potential as a low-power candidate for embedded devices. In federated learning scenarios, where multiple energy-constrained devices collaborate, adopting efficient SNN models with effective training methods is critical. However, research on training SNNs within federated learning systems is still very limited, particularly in terms of how to achieve both energy efficiency and robustness under system heterogeneity. This gap presents a significant opportunity for further exploration of SNNs in distributed learning settings. In this paper, we investigate a significant and innovative problem in robust spike-based federated learning, particularly in the presence of noise, and system heterogeneity. We majorly consider two types of system heterogeneity in this study, including data and client participation heterogeneity. To address this, we propose a novel federated learning framework, spike-based decoupled federated information-bottleneck learning (SDFIL), to enable robust, low-power federated learning through SNNs under system heterogeneity. Specifically, we design a decoupled information bottleneck principle tailored for local SNN training to maximize the mutual information between ground truth and model predictions while minimizing mutual information between intermediate representations. This method effectively minimizes the impact of outliers in non-independent and identically distributed (non-IID) data on model updates, thereby enhancing the performance of federated SNNs, resulting in enhanced robustness and reduced sensitivity to outliers. We evaluate the proposed SDFIL algorithm across a variety of settings, including different noise levels and varying degrees of system heterogeneity. The experimental results indicate that SDFIL demonstrates superior robustness compared to competing methods and generally achieves an improvement in overall accuracy of 5\% to 10\%. Additionally, it can achieve up to 7.7× higher energy efficiency compared to traditional artificial neural networks (ANNs).

Primary Area: applications to neuroscience & cognitive science

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.

Reciprocal Reviewing: I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 6758