Hyperbolic Deep Reinforcement Learning for Continuous ControlDownload PDF

01 Mar 2023 (modified: 29 May 2023)Submitted to Tiny Papers @ ICLR 2023Readers: Everyone
Keywords: deep reinforcement learning, hyperbolic representations, geometric deep learning, representation learning
TL;DR: We introduce a novel actor-critic architecture that leverages hyperbolic representations and demonstrate its effectiveness for continuous control.
Abstract: Integrating hyperbolic representations with Deep Reinforcement Learning (DRL) has recently been proposed as a promising approach for enhancing generalization and sample-efficiency in discrete control tasks. In this work, we extend hyperbolic RL to continuous control by introducing a novel hyperbolic actor-critic model. Empirically, our simple implementation outperforms its Euclidean counterpart, with significant gains on 16/24 tasks from the DeepMind Control Suite with pixel inputs. Notably, in the low-data regime, our method even outperforms several pre-trained unsupervised RL agents. Our findings show that hyperbolic representations provide a valuable inductive bias for continuous control.
5 Replies