TSkips: Efficiency Through Explicit Temporal Delay Connections in Spiking Neural Networks

Prajna G. Malettira, Shubham Negi, Wachirawit Ponghiran, Kaushik Roy

Abstract: Spiking Neural Networks (SNNs) with their bio-inspired Leaky Integrate-and-Fire (LIF) neurons inherently capture temporal information. This makes them well-suited for sequential tasks like processing event-based data from Dynamic Vision Sensors (DVS) and event-based speech tasks. Harnessing the temporal capabilities of SNNs requires mitigating vanishing spikes during training, capturing spatio-temporal patterns and enhancing precise spike timing. To address these challenges, we propose _TSkips_, augmenting SNN architectures with forward and backward skip connections that incorporate explicit temporal delays. These connections capture long-term spatio-temporal dependencies and facilitate better spike flow over long sequences. The introduction of _TSkips_ creates a vast search space of possible configurations, encompassing skip positions and time delay values. To efficiently navigate this search space, this work leverages training-free Neural Architecture Search (NAS) to identify optimal network structures and corresponding delays. We demonstrate the effectiveness of our approach on four event-based datasets: DSEC-flow for optical flow estimation, DVS128 Gesture for hand gesture recognition and Spiking Heidelberg Digits (SHD) and Spiking Speech Commands (SSC) for speech recognition. Our method achieves significant improvements across these datasets: up to 18% reduction in Average Endpoint Error (AEE) on DSEC-flow, 8% increase in classification accuracy on DVS128 Gesture, and up to ~8% and ~16% higher classification accuracy on SHD and SSC, respectively.

Submission Length: Long submission (more than 12 pages of main content)

Changes Since Last Submission:

We have made the minor revisions requested by the AE on scalability of the TSkips architectural modification (Section 4.6). We also provide an analysis on the robustness of NAS when searching for optimal TSkips configurations across random seeds, as detailed in the rebuttal (Appendix D.2).

Assigned Action Editor: Yunhe Wang

Submission Number: 4409