Training High Performance Spiking Neural Network by Temporal Model Calibration

Jiaqi Yan, Changping Wang, De Ma, Huajin Tang, Qian Zheng, Gang Pan

Abstract: Spiking Neural Networks (SNNs) are considered promising energy-efficient models due to their dynamic capability to process spatial-temporal spike information. Existing work has demonstrated that SNNs exhibit temporal heterogeneity, which leads to diverse outputs of SNNs at different time steps and has the potential to enhance their performance. Although SNNs obtained by direct training methods achieve state-of-the-art performance, current methods introduce limited temporal heterogeneity through the dynamics of spiking neurons or network structures. They lack the improvement of temporal heterogeneity through the lens of the gradient. In this paper, we first conclude that the diversity of the temporal logit gradients in current methods is limited. This leads to insufficient temporal heterogeneity and results in temporally miscalibrated SNNs with degraded performance. Based on the above analysis, we propose a Temporal Model Calibration (TMC) method, which can be seen as a logit gradient rescaling mechanism across time steps. Experimental results show that our method can improve the temporal logit gradient diversity and generate temporally calibrated SNNs with enhanced performance. In particular, our method achieves state-of-the-art accuracy on ImageNet, DVSCIFAR10, and N-Caltech101. Codes are available at https://github.com/zju-bmi-lab/TMC.

Lay Summary: Spiking Neural Networks (SNNs) are special types of AI models that are very energy-efficient and good at handling information that changes over time. After capturing dynamic features, SNNs will produce different outputs at different time steps, which is great for tasks like recognizing moving objects. However, we discovered that the current methods don't create enough diversity in how SNNs change over time. This results in SNNs that don't perform as well as they could. To address this, we proposed a method called Temporal Model Calibration (TMC), which helps improve the diversity of how SNNs change over time. Our experiments showed that TMC can enhance the performance of SNNs, making them more accurate on tasks like image recognition. Specifically, our method achieved top accuracy on datasets like ImageNet, DVSCIFAR10, and N-Caltech101. The codes for our method are available at https://github.com/zju-bmi-lab/TMC.

Primary Area: Deep Learning->Sequential Models, Time series

Keywords: Spiking Neural Networks, Direct Training, Temporal Heterogeneity, Temporal Model Calibration

Submission Number: 14510