Place Field Representation Learning During Policy Learning

M Ganesh Kumar, Blake Bordelon, Jacob A Zavatone-Veth, Cengiz Pehlevan

Track: long paper (up to 10 pages)

Domain: neuroscience

Abstract: As rodents navigate in a novel environment, a high place field density emerges at reward locations, fields elongate against the trajectory, and individual fields change spatial selectivity while demonstrating stable behavior. Why place fields demonstrate these characteristic phenomena during learning remains elusive. We develop a normative framework using reinforcement learning, whereby the Temporal Difference (TD) error modulates place field representations to improve policy learning. Place fields are modeled using Gaussian radial basis functions to represent spatial information, and directly synapse to an actor and critic for policy learning. Each field's amplitude, center, and width, as well as downstream weights, are updated online at each time step to maximize reward dependent objective. We demonstrate that this framework unifies three disparate phenomena observed in navigation experiments. Furthermore, we show that these place field representations improve policy convergence when learning to navigate to a single target and relearning new targets. To conclude, we develop a normative model that recapitulates several aspects of hippocampal place field learning dynamics and unifies mechanisms to offer testable predictions for future experiments.

Submission Number: 6