3DSMT: A Hybrid Spiking Mamba-Transformer for Point Cloud Analysis

Zhouzhiming, Yong He, Qiaoyun Wu, Chaoxu Mu, Ajmal Saeed Mian

Keywords: Point Cloud Analysis, Spiking neural network, Spiking Local Offset Attention, Spiking Mamba Block

Abstract:

The sparse unordered structure of point clouds causes unnecessary computation and energy consumption in deep models.
Conventionally, the Transformer architecture is leveraged to model global relationships in point clouds, however, its quadratic complexity restricts scalability. Although the Mamba architecture enables efficient global modeling with linear complexity, it lacks natural adaptability to unordered point clouds.
Spiking Neural Network (SNN) is an energy-efficient alternative to Artificial Neural Network (ANN), offering an ultra low-power event-driven paradigm.
The inherent sparsity and event-driven characteristics of SNN are highly compatible with the sparse distribution of point clouds. To balance efficiency and performance, we propose a hybrid spiking Mamba-Transformer (3DSMT) model for point cloud analysis. 3DSMT integrates a Spiking Local Offset Attention module to efficiently capture fine-grained local geometric features with a spiking Mamba block designed for unordered point clouds to achieve global feature integration with linear complexity. Experiments show that 3DSMT achieves state-of-the-art performance among SNN-based methods in shape classification, few-shot classification, and part segmentation tasks, significantly reducing computational energy consumption while also outperforming numerous ANN-based models.
Our source code is in supplementary material and will be made publicly available

Primary Area: applications to computer vision, audio, language, and other modalities

Submission Number: 7340