Exploiting MDP Symmetries for Offline Reinforcement Learning

Jinzhu Luo, Qi Zhang

Keywords: Offline RL, Symmetry

Abstract: Reinforcement Learning (RL) algorithms continue to face challenges in addressing out-of-distribution (OOD) issue in offline environments. One primary cause of such issue can be attributed to extrapolation error, which occur when an RL agent encounters actions that are not present in the offline dataset. In this study, we propose leveraging the inherent symmetry of the environment to expand the range of actions available for the agent's learning process. Our results demonstrate that by incorporating environmental symmetry, the performance and sample efficiency of basic RL algorithms can be improved in offline environment. This finding highlights the potential of harnessing environmental properties to enhance the generalization and robustness of offline RL algorithms.

Submission Number: 13