Implementing Inductive bias for different navigation tasks through diverse RNN attrractors

Tie XU, Omri Barak

Keywords: navigation, Recurrent Neural Networks, dynamics, inductive bias, pre-training, reinforcement learning

TL;DR: Task agnostic pre-training can shape RNN's attractor landscape, and form diverse inductive bias for different navigation tasks

Abstract: Navigation is crucial for animal behavior and is assumed to require an internal representation of the external environment, termed a cognitive map. The precise form of this representation is often considered to be a metric representation of space. An internal representation, however, is judged by its contribution to performance on a given task, and may thus vary between different types of navigation tasks. Here we train a recurrent neural network that controls an agent performing several navigation tasks in a simple environment. To focus on internal representations, we split learning into a task-agnostic pre-training stage that modifies internal connectivity and a task-specific Q learning stage that controls the network's output. We show that pre-training shapes the attractor landscape of the networks, leading to either a continuous attractor, discrete attractors or a disordered state. These structures induce bias onto the Q-Learning phase, leading to a performance pattern across the tasks corresponding to metric and topological regularities. Our results show that, in recurrent networks, inductive bias takes the form of attractor landscapes -- which can be shaped by pre-training and analyzed using dynamical systems methods. Furthermore, we demonstrate that non-metric representations are useful for navigation tasks.

Code: https://anonymous.4open.science/r/539372c8-c17b-4b48-a7da-56392ed685c4/