Learning State Reachability as a Graph in Translation Invariant Goal-based Reinforcement Learning Tasks

Hedwin BONNAVAUD, Alexandre Albore, Emmanuel Rachelson

Abstract:

Deep Reinforcement Learning proved efficient at learning universal control policies when the goal state is close enough to the starting state, or when the value function features few discontinuities.
But reaching goals that require long action sequences in complex environments remains difficult.
Drawing inspiration from the cognitive process which reuses learned atomic skills in a global planning procedure, we propose an algorithm which encodes reachability between abstract goals as a graph, and produces plans in this goal space.
Transitions between goals rely on the exploitation of a learned policy which enjoys a property we call \emph{translation invariant local optimality}, which encodes the intuition that goal-reaching skills can be reused throughout the state space.
Overall, our contribution permits solving large and difficult navigation tasks, outperforming related methods from the literature.

Submission Length: Long submission (more than 12 pages of main content)

Previous TMLR Submission Url: https://openreview.net/forum?id=p2E7P2Rsfv

Changes Since Last Submission: Removed the red text + changed the paper to camera ready.

Assigned Action Editor: ~Shixiang_Gu1

Submission Number: 2420