Discrete Planning with Neuro-algorithmic Policies

Marin Vlastelica Pogančić, Michal Rolinek, Georg Martius

Keywords: planning, imitation learning, reinforcement learning, combinatorics

TL;DR: We introduce neuro-algorithmic polices with the capability to plan. We show that these hybrid architectures facilitate considerable improvement in generalization in discrete control environments.

Abstract:

Although model-based and model-free approaches to learning the control of systems have achieved impressive results on standard benchmarks, most have been shown to be lacking in their generalization capabilities. These methods usually require sampling an exhaustive amount of data from different environment configurations.

We introduce a neuro-algorithmic policy architecture with the ability to plan consisting of a model working in unison with a shortest path solver to predict trajectories with low way-costs. These policies can be trained end-to-end by blackbox differentiation. We show that this type of architecture generalizes well to unseen environment configurations.