Feedback control guides credit assignment in recurrent neural networks

Klara Kaleb, Barbara Feulner, Juan A. Gallego, Claudia Clopath

Keywords: biologically-plausible learning, RNNs, motor control, feedback control

TL;DR: Feedback control may enable biological recurrent neural networks to achieve accurate and efficient credit assignment, facilitating real-time learning and adaptation in behavior generation.

Abstract:

How do brain circuits learn to generate behaviour?
While significant strides have been made in understanding learning in artificial neural networks, applying this knowledge to biological networks remains challenging.
For instance, while backpropagation is known to perform accurate credit assignment of error in artificial neural networks, how a similarly powerful process can be realized within the constraints of biological circuits remains largely unclear.
One of the major challenges is that the brain's extensive recurrent connectivity requires the propagation of error through both space and time, a problem that is notoriously difficult to solve in vanilla recurrent neural networks.
Moreover, the extensive feedback connections in the brain are known to influence forward network activity, but the interaction between feedback-driven activity changes and local, synaptic plasticity-based learning is not fully understood.
Building on our previous work modelling motor learning, this work investigates the mechanistic properties of pre-trained networks with feedback control on a standard motor task.
We show that feedback control of the ongoing recurrent network dynamics approximates the optimal first-order gradient with respect to the network activities, allowing for rapid, ongoing movement correction.
Moreover, we show that trial-by-trial adaptation to a persistent perturbation using a local, biologically plausible learning rule that integrates recent activity and error feedback is both more accurate and more efficient with feedback control during learning, due to the decoupling of the recurrent network dynamics and the injection of an adaptive, second-order gradient into the network dynamics.
Thus, our results suggest that feedback control may guide credit assignment in biological recurrent neural networks, enabling both rapid and efficient learning in the brain.

Primary Area: Neuroscience and cognitive science (neural coding, brain-computer interfaces)

Submission Number: 19940