Mimicking Human Intuition: Cognitive Belief-Driven Reinforcement Learning

Xingrui Gu, Guanren Qiao, Chuyi Jiang

Keywords: Reinforcement Learning, Cognitive Science, Smoothed Bellman Operator, Atari, Mujoco, Metadrive

Abstract: We propose an innovative framework inspired by cognitive principles: Cognitive Belief-Driven Reinforcement Learning (CBD-RL). Traditional reinforcement learning (RL) methods mainly rely on trial-and-error exploration, often lacking mechanisms to guide agents toward more informative decision-making and struggling to leverage past experiences, resulting in low sample efficiency. By incorporating cognitive heuristics, CBD-RL transforms conventional trial-and-error learning into a more structured and guided learning paradigm, simulating the human reasoning process. This framework's core is a belief system that optimizes action probabilities by integrating feedback with prior experience, thus enhancing decision making under uncertainty. It also organizes state-action pairs into meaningful categories, promoting generalization and improving sample efficiency. The concrete implementations of this framework, CBDQ, CBDPPO, and CBDSAC, demonstrate superior performance in both discrete and continuous action spaces in diverse environments such as Atari and MuJoCo. By bridging cognitive science and reinforcement learning, this research opens a new avenue for developing reinforcement learning systems that are more interpretable, efficient, and cognitively inspired.

Submission Number: 22