Learning Spatio-Temporal Representations Using Spike-Based Backpropagation

Deboleena Roy, Priyadarshini Panda, Kaushik Roy

Abstract: Spiking neural networks (SNNs) offer a promising alternative to current artificial neural networks to enable low-power event-driven neuromorphic hardware. However, training SNNs remains a challenge primarily because of the complex non-differentiable neuronal behavior arising from their spike-based computation. In this paper, we propose an algorithm to train spiking autoencoders on regenerative learning tasks. A sigmoid approximation is used in place of the Leaky Integrate-and-Fire neuron's threshold based activation during backpropagation to enable differentiability. The loss is computed on the membrane potential of the output layer, which is then backpropagated through the network at each time step. These spiking autoencoders learn meaningful spatio-temporal representations of the data, across two modalities - audio and visual. We demonstrate audio to image synthesis in a spike-based environment by sharing these spatio-temporal representations between the two modalities. These models achieve very low reconstruction loss, comparable to ANNs, on MNIST and Fashion-MNIST datasets, and while converting TI-46 digits audio samples to MNIST images.

Keywords: spiking neural networks, autoencoders, representation learning, backpropagation, multimodal