IncentRL: Bayesian Adaptation of Preference Gaps in Reinforcement Learning

ICLR 2026 Conference Submission21718 Authors

Keywords: Reinforcement Learning, Incentive Shaping, Bayesian Adaptation, Intrinsic Motivation, Exploration–Exploitation Trade-off

TL;DR: We introduce IncentRL: a cognitively inspired RL framework that adapts extrinsic & intrinsic rewards via Bayesian incentive shaping. Improves exploration efficiency & robustness on challenging exploration benchmarks like MiniGrid.

Abstract: Reinforcement learning agents often struggle in sparse-reward settings, where intrinsic signals such as curiosity or empowerment are used to aid exploration. Existing approaches typically rely on fixed trade-offs between extrinsic and intrinsic rewards, limiting adaptability across tasks. We introduce IncentRL, a cognitively inspired framework that unifies external rewards with internal preferences through adaptive incentive shaping. The central novelty is treating the incentive weight $\beta$ as a Bayesian random variable, updated online to balance exploration and exploitation without manual tuning. In addition, IncentRL augments task rewards with a KL-based penalty that aligns predicted outcome distributions with preferred outcomes. Theoretically, this connects to dopamine-based reward prediction error and the Free Energy Principle. Empirically, on MiniGrid and MountainCar, IncentRL improves sample efficiency and final performance over standard RL and fixed-regularization baselines. These results demonstrate that Bayesian adaptation of preference gaps removes the need for manual trade-off tuning, a core limitation of intrinsic motivation methods. Code is available at https://github.com/gravitywavelet/incentive-RL-anon.

Primary Area: reinforcement learning

Submission Number: 21718