What do RL Models Measure? Interpreting Model Parameters in Cognition and Neuroscience

Maria K Eckstein, Linda Wilbrecht, Anne Collins

Abstract: Reinforcement learning (RL) is a concept that has been invaluable to re-search elds including machine learning, neuroscience, and cognitive science.However, what RL entails partly diers between elds, leading to dicul-ties when interpreting and translating ndings. This paper lays out thesedierences and zooms in on cognitive (neuro)science, revealing that we of-ten overinterpret RL modeling results, with severe consequences for futureresearch. Specically, researchers often assume|implicitly|that model pa-rametersgeneralizebetween tasks, models, and participant populations, de-spite overwhelming negative empirical evidence for this assumption. Wealso often assume that parameters measure specic, unique, and meaningful(neuro)cognitive processes, a concept we callinterpretability, for which empir-ical evidence is also lacking. We conclude that future computational researchneeds to pay increased attention to these implicit assumptions when usingRL models, and suggest an alternative framework that resolves these issuesand allows us to unleash the potential of RL in cognitive (neuro)science.