Brain signals of a Surprise-Actor-Critic model: Evidence for multiple learning modules in human decision making

Vasiliki Liakoni, Marco P. Lehmann, Alireza Modirshanechi, Johanni Brea, Antoine Lutti, Wulfram Gerstner, Kerstin Preuschoff

Abstract: Highlights•Novel sequential learning task to dissociate model-free from model-based signals.•Human behavior best explained by the Actor-critic framework.•Approximate Bayesian approach for surprise-based learning of the world-model.•Neural signatures of a surprise-modulated Actor-critic algorithm.•Evidence for temporal-difference, policy gradient, and surprise in the human brain.