Neural networks with motivation

Sergey A. Shuvaev, Ngoc B. Tran, Marcus Stephenson-Jones, Bo Li, Alexei A. Koulakov

Keywords: neuroscience, brain, motivation, learning, reinforcement learning, recurrent neural network, deep learning

TL;DR: We developed an RL model of motivated behaviors, then trained both our model and lab mice on biologically realistic task to make and validate predictions about neuronal connectivity in mouse brain.

Abstract: How can animals behave effectively in conditions involving different motivational contexts? Here, we propose how reinforcement learning neural networks can learn optimal behavior for dynamically changing motivational salience vectors. First, we show that Q-learning neural networks with motivation can navigate in environment with dynamic rewards. Second, we show that such networks can learn complex behaviors simultaneously directed towards several goals distributed in an environment. Finally, we show that in Pavlovian conditioning task, the responses of the neurons in our model resemble the firing patterns of neurons in the ventral pallidum (VP), a basal ganglia structure involved in motivated behaviors. We show that, similarly to real neurons, recurrent networks with motivation are composed of two oppositely-tuned classes of neurons, responding to positive and negative rewards. Our model generates predictions for the VP connectivity. We conclude that networks with motivation can rapidly adapt their behavior to varying conditions without changes in synaptic strength when expected reward is modulated by motivation. Such networks may also provide a mechanism for how hierarchical reinforcement learning is implemented in the brain.