Difference Predictive Coding for Training Spiking Neural Networks

Ville Karlsson, Nicklas Fianda, Joni-Kristian Kämäräinen

Keywords: Spiking neural networks, predictive coding, biologically plausible learning, neuromorphic computing, difference predictive coding, local learning rules, energy efficiency, communication efficiency, spike-based learning, surrogate gradient alternatives

TL;DR: We create a spiking neural network compatible learning algorithm based on predictive coding theory

Abstract: Predictive coding networks (PCNs) offer a local-learning alternative to backpropagation in which layers communicate residual errors, aligning well with biological computation and neuromorphic hardware. In this work we introduce Difference Predictive Coding (DiffPC), a spike-native PC formulation for spiking neural networks. DiffPC replaces dense floating-point messages with sparse ternary spikes, provides spike-compatible target and error updates, and employs adaptive threshold schedules for event-driven operation. We validate DiffPC on fully connected and convolutional architectures, demonstrating competitive performance on MNIST (99.3\%) and Fashion-MNIST (89.6\%), and outperforming a backpropagation baseline on CIFAR-10. Crucially, this performance is achieved with high communication sparsity, reducing data movement by over two orders of magnitude compared to standard predictive coding. DiffPC thus establishes a faithful, hardware-aligned framework for communication-efficient training on neuromorphic platforms.

Primary Area: unsupervised, self-supervised, semi-supervised, and supervised representation learning

Submission Number: 11329