Hyperbolic Deep Reinforcement Learning for Continuous Control

Omar Salemohamed, Edoardo Cetin, Sai Rajeswar, Arnab Kumar Mondal

Keywords: deep reinforcement learning, hyperbolic representations, geometric deep learning, representation learning

TL;DR: We introduce a novel actor-critic architecture that leverages hyperbolic representations and demonstrate its effectiveness for continuous control.

Abstract: Integrating hyperbolic representations with Deep Reinforcement Learning (DRL) has recently been proposed as a promising approach for enhancing generalization and sample-efficiency in discrete control tasks. In this work, we extend hyperbolic RL to continuous control by introducing a novel hyperbolic actor-critic model. Empirically, our simple implementation outperforms its Euclidean counterpart, with significant gains on 16/24 tasks from the DeepMind Control Suite with pixel inputs. Notably, in the low-data regime, our method even outperforms several pre-trained unsupervised RL agents. Our findings show that hyperbolic representations provide a valuable inductive bias for continuous control.