A Parallel Multi-compartment Spiking Neuron For Multi-scale Sequential Modeling

Xinyi Chen, Jibin Wu, Chenxiang Ma, Yinsong Yan, KC Tan

Primary Area: applications to neuroscience & cognitive science

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Keywords: Spiking Neural Networks, Spiking Neuron Models, Multi-compartment Model, Sequential Modeling, Brain-inspired Computing, Neuromorphics.

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.

TL;DR: We proposed a bio-inspired spiking neuron model to tackle the inherent memory constraints and slow training speed issues in existing neuron models, enhancing spiking neural networks' ability to preserve and integrate multi-scale temporal information.

Abstract: The human brain possesses remarkable abilities in processing sensory signals that exhibit complex temporal dynamics. However, brain-inspired Spiking Neural Networks (SNNs) encounter challenges when dealing with sensory signals that have a high temporal complexity. These challenges primarily arise from the utilization of simplified spiking neuron models, such as the widely adopted Leaky Integrate-and-Fire (LIF) model, which has limited capability to process temporal information across multiple time scales. Additionally, these spiking neuron models can only be updated sequentially in time, resulting in slow training processes that pose particular difficulties when dealing with long sequences. To address these issues, we propose a novel Parallel Multi-compartment Spiking Neuron (PMSN), which is derived from the cable model of hippocampus pyramidal neurons. The PMSN model captures the intricate interactions among various neuronal compartments, allowing multi-scale temporal information to be preserved and integrated for effective sequential modeling. Furthermore, the PMSN model has been meticulously designed to facilitate parallel training on GPU-accelerated machine learning frameworks. Our experimental results across numerous sequential modeling tasks demonstrate the superior performance of the proposed PMSN model compared with other spiking neuron models. Specifically, it exhibits enhanced classification accuracy, accelerated simulation, and favorable trade-offs between accuracy and computation cost.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 5407