A role for phasic serotonergic signaling in regulating augmentations during open-ended learning
Track: Full track
Keywords: data augmentation, open-ended learning, zero-shot generalization, serotonin
TL;DR: serotonin-like state prediction errors can effectively regulate data augmentations online during open-ended learning
Abstract: Humans and animals need to rapidly adapt to dynamically changing environments given only few experiences while high-performance artificial systems require large datasets. In order to bridge this gap, we consider data augmentations, which have been shown to substantially improve the data-efficiency and generalization capabilities of many machine learning models including reinforcement learning agents. However, how augmentation should be coordinated online during open-ended interactions with the world is unclear. We take inspiration from the brain in addressing this issue. Encountering unexpected environment states (signaled by state prediction errors, SPEs) has been associated with the phasic release of serotonin, a neurotransmitter known to mediate cognitive flexibility in humans. We hypothesize that serotonin triggers augmentations and that this facilitates adaptation to novel environments. In our agent-based simulations, learning from augmentations improves state-prediction in unfamiliar contexts within a minimal circular environment and in gridworlds. Furthermore, we find that augmentations timed to high SPEs are particularly effective. These preliminary results are consistent with a functional role for serotonergic neuromodulation in open-ended adaptation of natural and artificial systems based on regulating the augmentation of experience.
Submission Number: 33
Loading