Unlocking Long-Term Dependencies in Spiking Neural Networks with a Recurrent LIF Memory Module

ICLR 2026 Conference Submission18405 Authors

Keywords: Spiking Neural Networks, Long-Term Dependencies, Recurrent Dynamics, Neuromorphic Computing, Sequence Modeling

Abstract: Processing long sequence data such as speech requires models to maintain long-term dependencies, which is challenging for recurrent spiking neural networks (RSNNs) due to high temporal dynamics in neuron models that leaks stored information in their membrane potentials, and faces vanishing gradients during back-propagation through time. These issues can be mitigated by employing more complex neuron designs, such as ALIF and TC-LIF, but these neuron-level solutions often incur high computational costs and complicate hardware implementation, undermining the efficiency advantages of SNNs. Here we propose a network-level solution that leverages the dynamical interactions of a few LIF neurons to enhance long-term information storage. The memory capability of this LIF-based micro-circuits is adaptively modulated by global recurrent connections of the RSNN, contributing to selective enhancement of temporal information retention, and ensures stable gradient gain when propagation through time. The proposed model outperforms previous methods including LSTM, ALIF, and TC-LIF in long sequence tasks, achieving 96.52\% accuracy on the PS-MNIST dataset. Furthermore, our method also provides a compelling efficiency advantage, yielding up to 400× improvement compared to conventional models such as LSTM. This work paves the way for building cost-effective, hardware-friendly, and interpretable spiking neural networks for long sequence modeling.

Primary Area: learning on time series and dynamical systems

Submission Number: 18405