Learning State Reachability as a Graph in Translation Invariant Goal-based Reinforcement Learning Tasks
Abstract: Deep Reinforcement Learning proved efficient at learning universal control policies when the goal state is close enough to the starting state, or when the value function features few discontinuities.
But reaching goals that require long action sequences in complex environments remains difficult.
Drawing inspiration from the cognitive process which reuses learned atomic skills in a global planning procedure, we propose an algorithm which encodes reachability between abstract goals as a graph, and produces plans in this goal space.
Transitions between goals rely on the exploitation of a learned policy which enjoys a property we call \emph{translation invariant local optimality}, which encodes the intuition that goal-reaching skills can be reused throughout the state space.
Overall, our contribution permits solving large and difficult navigation tasks, outperforming related methods from the literature.
Submission Length: Long submission (more than 12 pages of main content)
Previous TMLR Submission Url: https://openreview.net/forum?id=p2E7P2Rsfv
Changes Since Last Submission: Following the recommendation of the action editor and, as requested by the reviewers and the action editor, we have:
- included all experiments with DSG,
- rephrased the paper (mainly the introduction, algorithm presentation and experimental results) to make the claims more specific,
- restructured the paper for better readability.
Assigned Action Editor: ~Shixiang_Gu1
Submission Number: 2420
Loading