Spatio-Temporal Dependency-Aware Neuron Optimization for Spiking Neural Networks
Keywords: Neuromorphic computation, spiking neural networks, temporal dependency
TL;DR: We propose a neuron design and search method to adaptively optimize temporal dependencies in SNNs, achieving superior performance across various datasets.
Abstract: As a biologically inspired computing paradigm, Spiking Neural Networks (SNNs) process information through discrete spike sequences, mimicking the brain's temporal dynamics and energy efficiency. The combination of backpropagation through time (BPTT) and direct input encoding (i.e., feeding decimal data directly into the network) has emerged as the mainstream training approach for SNNs. However, this combination introduces varying temporal dependency requirements across the network’s spatial dimension. These differences are often neglected in existing studies, which typically apply uniform temporal dependency configurations throughout the network. Consequently, this could result in missing key gradients or introducing redundant ones in the temporal dimension, ultimately affecting the network's performance. To address this gap, we propose a novel Spatio-Temporal Dependency-Aware Neuron Optimization (ST-DANO) method for SNNs, which consists of two key components: neuron design and neuron search. Specifically, to overcome the limitations of traditional Leaky Integrate-and-Fire (LIF) neurons in adapting to varying temporal dependencies, we designed two variants, Long-LIF and Short-LIF, which improve the neuron's ability to capture long-term and short-term dependencies, respectively, by dynamic modulation of membrane potential thresholds and time constants. After validating our neuron designs through ablation studies, we developed a layer-wise neuron search strategy that automatically selects the optimal neuron type for each layer to ensure optimal temporal dependency configurations across the network. Extensive experiments on static and neuromorphic datasets demonstrate that ST-DANO can effectively adapt to temporal dependency differences across the spatial dimension in SNNs under various time-step configurations. The resulting architectures surpass state-of-the-art performance, achieving a remarkable 83.90\% accuracy on the DVS-CIFAR-10 dataset—a more than 5\% improvement over the baseline.
Supplementary Material: zip
Primary Area: optimization
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Submission Number: 8134
Loading