Progressively Efficient Communication

Khanh Nguyen, Ruijie Zheng, Hal Daumé III, Furong Huang, Karthik Narasimhan

Keywords: interactive learning, human-ai communication

TL;DR: A framework for learning progressively efficiently

Abstract:

Assistant AI agents should be capable of rapidly acquiring novel skills and adapting
to new user preferences. Traditional frameworks like imitation learning and
reinforcement learning do not facilitate this capability because they support only
low-level, inefficient forms of communication. In contrast, humans communicate
with progressive efficiency by defining and sharing abstract intentions. Reproducing
similar capability in AI agents, we develop a novel learning framework named
Communication-Efficient Interactive Learning (CEIL). By equipping a learning
agent with an abstract, dynamic language and an intrinsic motivation to learn
with minimal communication effort, CEIL leads to emergence of a human-like
pattern where the learner and the teacher communicate progressively efficiently by
exchanging increasingly more abstract intentions. CEIL demonstrates impressive
performance and communication efficiency in a 2D MineCraft domain featuring
long-horizon decision-making tasks. Agents trained with CEIL quickly master
new tasks, outperforming non-hierarchical and hierarchical imitation learning by
up to 50% and 20% in absolute success rate, respectively, given the same number
of interactions with the teacher. Especially, the framework performs robustly with
teachers modeled after human pragmatic communication behavior.

Submission Number: 7