Human-Allied Relational Reinforcement Learning

Fateme Golivand Darvishvand, Hikaru Shindo, Sahil Sidheekh, Kristian Kersting, Sriraam Natarajan

Keywords: Relational Reinforcement Learning, Active Advice

Abstract: Reinforcement learning (RL) has experienced a second wind in the past decade. While incredibly successful in images and videos, these systems still operate within the realm of propositional tasks ignoring the inherent structure that exists in the problem. Consequently, relational extensions (RRL) have been developed for such structured problems that allow for effective generalization to arbitrary number of objects. However, they inherently make strong assumptions about the problem structure. We introduce a novel framework that combines RRL with object-centric representation to handle both structured and unstructured data. We enhance learning by allowing the system to actively query the human expert for guidance by explicitly modeling the uncertainty over the policy. Our empirical evaluation demonstrates the effectiveness and efficiency of our proposed approach.

Paper Track: Technical paper

Submission Number: 53