Advancing Sparse Attention in Spiking Neural Networks via Spike-Timing-Based Prioritization

Minsuk Jang

Keywords: Spiking Neural Networks, Neuromorphic Computing, Transformer, Energy Efficiency, Sparse Attention, Temporal Coding, Event-based Vision

TL;DR: SPARTA leverages biologically-inspired spike timing cues (firing rate, spike timing, intervals) to enable sparse attention in spiking transformers, achieving 65.4% sparsity and competitive accuracy while preserving neuromorphic principles.

Abstract: Current Spiking Neural Networks (SNNs) underutilize the temporal dynamics inherent in spike-based processing, relying primarily on rate coding while overlooking precise timing information that provides rich computational cues. We address this by proposing \textbf{SPARTA} (Spiking Priority Attention with Resource-Adaptive Temporal Allocation), which leverages heterogeneous neuron dynamics and spike-timing information to enable sparse attention mechanisms. SPARTA extracts temporal cues—including firing patterns, spike timing, and inter-spike intervals—to prioritize tokens for processing, achieving 65.4% sparsity through competitive gating. By selecting only the most salient tokens, SPARTA reduces attention complexity from (O(N^{2})) to (O(K^{2})), where (k!\ll!n). Our approach achieves state-of-the-art accuracy on DVS-Gesture (98.78%) and competitive performance on CIFAR10-DVS (83.06%) and CIFAR-10 (95.3%), demonstrating that spike-timing utilization enables both computational efficiency and competitive accuracy.

Primary Area: applications to computer vision, audio, language, and other modalities

Submission Number: 6912