State-Self-Determining Reinforcement Learning for Adapting to Unknown Environments and Elucidating Higher Brain Functions

Kazuhiro Sakamoto, Tokio Katakura, Naoki M. Tamura, Nao Matsumoto

Abstract: The real world is an indefinite environment in which the probability space is not specified in advance. Here, we developed a learning model in which the state space is expanded so that it refers to the arbitrary length of previous states, based on two criteria: experience saturation and decision uniqueness of action selection. The model was tested by behavioral tasks called a two-target search task and action sequence task that have contributed to elucidating of functional roles of higher motor areas in the cerebral cortex. The proposed state-self-determining reinforcement learning model serves as a basis not only for high adaptability to an indefinite environment but also for building a computational theory of higher brain functions.