A biologically-plausible alternative to backpropagation using pseudoinverse feedback

Mia Cameron, Yusi Chen, Terrence Sejnowski

Keywords: biologically-plausible learning rules, Newton-like methods, local learning rules

TL;DR: We show how pseudoinverse, reciprocal feedback connections between layers can facilitate a biologically-plausible alternative to backpropagation in neural networks.

Abstract: Despite its successes in both practical machine learning and neural modeling, the backpropagation algorithm has long been considered biologically implausible (Crick, 1989). Previous solutions to this biological implausibility have proposed the existence of a separate, error feedback network, in which error at the final layer may be propagated backwards to earlier layers in a manner similar to backpropagation. However, biological evidence suggests that feedback connections in the cortex may function more similarly to an autoencoder, rather than being exclusively used as error feedback (Marino, 2020; Chen et al., 2024). Here, we attempt to unify these two paradigms by showing how autoencoder-like, inverse feedback connections may be used to minimize error throughout a feedforward neural network. Our proposed mechanism, Reciprocal Feedback, consists of two contributions: first we show how a modification of the Recirculation algorithm (Hinton & McClelland, 1988) is capable of learning the Moore-Penrose pseudoinverse of a pair of network weights. Then, we will show how, using a Newton-like method (Hildebrandt & Graves, 1927), locally-learned pseudoinverse feedback connections may be used to facilitate an alternative optimization method to traditional gradient descent - while alleviating the need to compute the weight transpose, or use direct feedback connections from the final layer. In the MNIST and CIFAR-10 classification tasks, our method obtains an asymptotic error similar to backpropagation, in fewer iterations than comparable biologically-plausible algorithms, such as Feedback Alignment (Lillicrap et al., 2014).

Primary Area: applications to neuroscience & cognitive science

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 11720