Graph-Theoretic Intrinsic Reward: Guiding RL with Effective Resistance

ICLR 2026 Conference Submission10814 Authors

Keywords: Reinforcement Learning, Intrinsic Motivation, Goal Conditioned RL, Effective Resistance

TL;DR: We propose an intrinsic reward formulation using the notion of Effective Resistance based on spectral graph theory, for learning robust policies in sparse environments.

Abstract: Exploration of dynamic environments with sparse rewards is a significant challenge in Reinforcement Learning, often leading to inefficient exploration and brittle policies. To address this, we introduce a novel graph-based intrinsic reward using Effective Resistance, a metric from spectral graph theory. This reward formulation guides the agent to seek configurations that are directly correlated to successful goal reaching states. We provide theoretical guarantees, proving that our method not only learns a robust policy but also achieves faster convergence by serving as a variance reduction baseline to the standard discounted reward formulation. We perform extensive empirical analysis across several challenging environments to demonstrate that our approach significantly outperforms state-of-the-art baselines, demonstrating improvements of up to 59% in success rate, 56% in timesteps taken to reach the goal, and 4 times more accumulated reward. We augment all of the supporting lemmas and theoretically motivated hyperparameter choices with corresponding experiments.

Primary Area: reinforcement learning

Submission Number: 10814