Training Spiking Neural Networks via Augmented Direct Feedback Alignment

Yongbo Zhang, Katsuma Inoue, Mitsumasa Nakajima, Toshikazu Hashimoto, Yasuo Kuniyoshi, Kohei Nakajima

Keywords: Spiking neural networks, neuromorphic computing, learning algorithm

TL;DR: We systematically studied a backpropagation-free and gradient-free learning algorithm, aDFA, on SNNs, achieved good performance without accurate prior knowledge of the utilized system, which is highly preferred for physical implementation.

Abstract:

Spiking neural networks (SNNs), the models inspired by the mechanisms of real neurons in the brain, transmit and represent information by employing discrete action potentials or spikes. The sparse, asynchronous properties of information processing make SNNs highly energy efficient, leading to SNNs being promising solutions for implementing neural networks in neuromorphic devices.
However, the nondifferentiable nature of SNN neurons makes it a challenge to train them.
The current training methods of SNNs that are based on error backpropagation (BP) and precisely designing surrogate gradient are difficult to implement and biologically implausible, hindering the implementation of SNNs on neuromorphic devices.
Thus, it is important to train SNNs with a method that is both physically implementatable and biologically plausible.
In this paper, we propose using augmented direct feedback alignment (aDFA), a gradient-free approach based on random projection, to train SNNs.
This method requires only partial information of the forward process during training, so it is easy to implement and biologically plausible.
We systematically demonstrate the feasibility of the proposed aDFA-SNNs scheme, propose its effective working range, and analyze its well-performing settings by employing genetic algorithm. We also analyze the impact of crucial features of SNNs on the scheme, thus demonstrating its superiority and stability over BP and conventional direct feedback alignment.
Our scheme can achieve competitive performance without accurate prior knowledge about the utilized system, thus providing a valuable reference for physically training SNNs.

Submission Number: 10