Generalizable Imitation Learning from Observation via Inferring Goal Proximity

Youngwoon Lee, Andrew Szot, Shao-Hua Sun, Joseph J Lim

Keywords: Imitation Learning, Learning from Observation

Abstract: Task progress is intuitive and readily available task information that can guide an agent closer to the desired goal. Furthermore, a task progress estimator can generalize to new situations. From this intuition, we propose a simple yet effective imitation learning from observation method for a goal-directed task using a learned goal proximity function as a task progress estimator for better generalization to unseen states and goals. We obtain this goal proximity function from expert demonstrations and online agent experience, and then use the learned goal proximity as a dense reward for policy training. We demonstrate that our proposed method can robustly generalize compared to prior imitation learning methods on a set of goal-directed tasks in navigation, locomotion, and robotic manipulation, even with demonstrations that cover only a part of the states.

Code Of Conduct: I certify that all co-authors of this work have read and commit to adhering to the NeurIPS Statement on Ethics, Fairness, Inclusivity, and Code of Conduct.

Code: https://clvrai.com/gpil

TL;DR: We propose a simple yet effective imitation learning from observation method using a task progress estimator (goal proximity function) for better generalization to unseen states and goals.