Forward Target Propagation: A Forward-Only Approach to Global Error Credit Assignment via Local Losses

TMLR Paper9152 Authors

Abstract:

Training neural networks has traditionally relied on backpropagation (BP), a gradient-based
algorithm that, despite its widespread success, suffers from key limitations in both biological
and hardware perspectives. These include backward error propagation by symmetric weights,
non-local credit assignment, update locking, and frozen activity during backward passes. We
propose Forward Target Propagation (FTP), a biologically plausible and computationally
efficient alternative that replaces the backward pass with a second forward pass. FTP
estimates layer-wise targets using only feedforward computations, eliminating the need for
symmetric feedback weights or learnable inverse functions, hence enabling modular and local
learning. We evaluate FTP on fully connected networks, CNNs, and RNNs, demonstrating
accuracies competitive with BP on MNIST, CIFAR-10, and CIFAR-100, as well as effective
modeling of long-term dependencies in sequential tasks. FTP shows improved robustness
under quantized low-precision and emerging hardware constraints while also demonstrating
substantial efficiency gains over other biologically inspired methods such as target propagation
variants and forward-only learning algorithms. With its minimal computational overhead,
forward-only nature, and hardware compatibility, FTP provides a promising direction for
energy-efficient on-device learning and neuromorphic computing.

Submission Type: Regular submission (no more than 12 pages of main content)

Assigned Action Editor: ~Joao_Sacramento1

Submission Number: 9152