A Geometric Lens on RL Environment Complexity Based on Ricci Curvature

Ali Saheb Pasand, Pablo Samuel Castro, Pouya Bashivan

Keywords: Information Geometry, Complexity, Intrinsic Reward

TL;DR: This paper uses Ollivier-Ricci Curvature to reveal local RL task complexity and shows it improves exploration when used as an intrinsic reward.

Abstract: We introduce Ollivier-Ricci Curvature (ORC) as an information-geometric tool for analyzing the local structure of reinforcement learning (RL) environments. We establish a novel connection between ORC and the Successor Representation (SR), enabling a geometric interpretation of environment dynamics decoupled from reward signals. Our analysis shows that states with positive and negative ORC values correspond to regions where random walks converge and diverge respectively, which are often critical for effective exploration. ORC is highly correlated with established environment complexity metrics, yet integrates naturally with standard RL frameworks based on SR and provides both global and local complexity measures. Leveraging this property, we propose an ORC-based intrinsic reward that guides agents toward divergent regions and away from convergent traps. Empirical results demonstrate that our curvature-driven reward substantially improves exploration performance across diverse environments, outperforming both random and count-based intrinsic reward baselines.

Submission Number: 7