Learning State Reachability as a Graph in Translation Invariant Goal-based Reinforcement Learning Tasks
Abstract: Deep Reinforcement Learning proved efficient at learning universal control policies when the goal state is close enough to the starting state, or when the value function features few discontinuities. But reaching goals that require long action sequences in complex environments remains difficult. Drawing inspiration from the cognitive process which reuses learned atomic skills in a global planning procedure, we propose an algorithm which encodes reachability between abstract goals as a graph, and produces plans in this goal space. Transitions between goals rely on the exploitation of a learned policy which enjoys a property we call translation invariant local optimality, which encodes the intuition that goal-reaching skills can be reused throughout the state space. Overall, our contribution permits solving large and difficult navigation tasks, outperforming related methods from the literature.
Submission Length: Regular submission (no more than 12 pages of main content)
Previous TMLR Submission Url: https://openreview.net/forum?id=NkahJNzNqo
Changes Since Last Submission: Made modifications in the paper's text (visible in red to facilitate reading) to account for all reviewers' comments.
Assigned Action Editor: ~Jiajun_Wu1
Submission Number: 1220
Loading