Decide When Ready: Stepwise Incremental Inference with Early-Exit in Spiking Neural Networks

Yan Zhang, Dawei Ren, Hongying Tang, Qiaoling Zhou, Ronghua Qin

Keywords: Spiking neural networks, neuromorphic computing, incremental inference, casual time attention, early exit

TL;DR: We propose an incremental inference architecture for spiking neural networks that updates predictions online from event streams, reducing latency and redundancy, and is well-suited for neuromorphic edge deployment.

Abstract: Spiking Neural Networks (SNNs) are well-suited for low-power, low-latency dynamic visual perception due to their event-driven computation. However, existing SNNs rely on fixed time steps for training and inference, which leads to buffering requirements and mismatches with neuromorphic hardware, thus neglecting the potential for early recognition using partial event streams. In neuromorphic computing, ideal dynamic visual perception should be event-driven, with models continuously updating states based on incoming events and producing results as soon as confidence criteria are met. To address this, we propose the Spiking Incremental Recognition Network (SIREN), an incremental inference architecture designed to approximate this ideal paradigm. During training, the model processes event streams in fixed steps, while at inference it processes event frames step by step, updating states continuously and making dynamic decisions. SIREN integrates multiple spiking neuron types and a Spiking State-Space Model (S-SSM) to capture multiscale temporal dependencies. It also combines Causal Time Self-Attention (CTSA) with early-exit strategies for efficient termination. We evaluate our approach on three Dynamic Vision Sensor (DVS) datasets, achieving state-of-the-art performance in recognition tasks, including SL-Animals-DVS, DVS128-Gesture and the THU-EACT-50 subset, with accuracies of 93.33%, 97.92% and 100% respectively. Concurrently, we reduce the average inference steps from 16 to 9.5, with fewer synaptic operations (SOPs), demonstrating its potential for resource-constrained event-based recognition.

Primary Area: learning on time series and dynamical systems

Submission Number: 2863