Exact Online Learning with Gamma-memory delays for Accurate Feedforward SNNs

Roel Koopman, Sebastian Otte, Sander Bohte

Keywords: spiking neural network, online learning, delay learning, energy-efficiency, feedforward

TL;DR: The SpikingGamma model enables exact online training of feedforward SNNs that can capture precise temporal patterns and achieve competitive accuracy on complex tasks while being insensitive to the temporal precision of the simulation.

Abstract: Spiking Neural Networks (SNNs) promise energy-efficient, low-latency AI through sparse, event-driven computation. Neuromorphic hardware can realize this efficiency by exploiting high temporal resolution, as precise spike timing supports compact and sparse information processing. Yet, training SNNs under fine temporal discretization remains a major challenge. In state-of-the-art approaches, spiking neurons are modeled as self-recurrent units, embedded into recurrent networks to maintain state, and trained with BPTT or RTRL variants based on surrogate gradients. We show that these methods scale poorly with temporal resolution, while online approximations methods are inherently unstable. We solve this problem by developing recursive memory structures combined with a linear–nonlinear interpretation of spike-train generation in spiking neurons: the SpikingGamma model. We show that SpikingGamma models support direct error backpropagation without surrogate gradients, can learn fine temporal patterns with minimal spiking in an online manner, and scale feedforward SNNs to complex tasks with competitive accuracy, all while being insensitive to the temporal precision of the model. Our approach offers both an alternative to current recurrent SNNs trained with surrogate gradients, and a direct route for mapping SNNs to neuromorphic hardware.

Primary Area: unsupervised, self-supervised, semi-supervised, and supervised representation learning

Submission Number: 12183