Conversion of sparse Artificial Neural Network to sparse Spiking Neural Network can save up to 99% of energy

ICLR 2026 Conference Submission16520 Authors

Keywords: Spiking neural networks, ANN-to-SNN conversion dynamic sparse training, Cannistraci-Hebb Training, sustainable AI, energy-efficient architectures

Abstract: Artificial Neural Networks (ANNs) are becoming increasingly important but face the challenge of the large scale and high energy consumption. Dynamic Sparse Training (DST) aims to reduce the memory and energy consumption of ANNs by learning sparse network topologies, which ultimately results in structural connection sparsity. Meanwhile, Spiking Neural Networks (SNNs) have attracted increasing attention due to their biological plausibility and event-driven nature, which ultimately results in temporal sparsity. To bypass the difficulty of directly training SNNs, converting pre-trained ANNs to SNNs (ANN2SNN) is becoming a popular approach to obtain high-performance SNNs. Here for the first time, we investigated the advantage of dynamically spare trained ANNs for conversion into sparse SNNs. By adopting Cannistraci-Hebb Training (CHT), a state-of-the-art brain-inspired DST family that resembles synaptic turnover during neuronal connectivity learning in brain circuits, we examined the extent to which connectivity sparsity impacts the accuracy and energy efficiency of SNNs across different conversion approaches. The results show that sparse SNNs can achieve accuracy comparable to or even surpassing that of dense SNNs. Moreover, sparse SNNs can reduce energy consumption by up to 99% compared with dense SNNs. Furthermore, driven by the interest in understanding the physical dynamic interactions between firing rate and accuracy in SNNs, we systematically analyzed the temporal relationship between the saturation of firing rate and accuracy in SNNs. Our results reveal a significant time lag where firing rate saturation precedes accuracy saturation. We also demonstrate that the magnitude of this time lag is significantly different between sparse and dense networks, where the average time lag of sparse SNNs is higher than that of dense SNNs. By combining the structural sparsity of DST and temporal sparsity of SNNs, we make a step forward to the brain-like computational network architecture with high performance and energy efficiency.

Primary Area: applications to neuroscience & cognitive science

Submission Number: 16520