Spiking Transformer-CNN for Event-based Object Detection

Dongyang Ma, Jianing Li, Zhengyu Ma, Wei Zhang, Chenlin Zhou, Yonghong Tian

Keywords: Event data, Object detection, Spike neural networks, Low power consumption, Transformer-CNN

Abstract: Spiking Neural Networks (SNNs) enable energy-efficient computation through event-driven computing and multiplication-free inference, making them well-suited for processing sparse events. Recently, deep Spiking Convolutional Neural Networks (CNNs) have shown energy efficiency advantages on event-based object detection. However, spiking CNNs have been limited to local and single-scale features, making it challenging for them to achieve better detection accuracy. To address this challenge, we propose a hierarchical Spiking Transformer-CNN (i.e., Spike-TransCNN) architecture, which is the first attempt to leverage the global information extraction capabilities of Spiking Transformers and the local information capture abilities of Spiking CNNs for event-based object detection. Technically, we first propose using the Spiking Transformer to extract global features and employ a multi-scale local feature extraction CNN module to complement the Spiking Transformers in local feature extraction. Then, we design intra-stage and inter-stage feature fusion modules to integrate global and multi-scale local features within the network architecture. Experimental results demonstrate that our Spike-TransCNN significantly outperforms existing SNN-based object detectors on the Gen1 dataset, achieving higher detection accuracy (mAP 0.336 vs. 0.321) with lower energy consumption (5.49 mJ vs. 7.26 mJ). Our code can be available in the supplementary materials.

Primary Area: applications to computer vision, audio, language, and other modalities

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 3673