A Unifying Perspective on Language-Based Task Representations for Robot Control

Maximilian Tölle, Boris Belousov, Jan Peters

Keywords: Language-Based Task Representations, Robot Control

TL;DR: This paper proposes a taxonomy to categorize the landscape of existing methods that condition robot action generation on natural language commands.

Abstract: Natural language is becoming increasingly important in robot control for both high-level planning and goal-directed conditioning of motor skills. While a number of solutions have been proposed already, it is yet to be seen what architecture will succeed in seamlessly integrating language, vision, and action. To better understand the landscape of existing methods, we propose to view the algorithms from the perspective of “Language-Based Task Representations”, i.e., categorizing the methods that condition robot action generation on natural language commands according to their task representation and embedding architecture. Our proposed taxonomy intuitively groups existing algorithms, highlights their commonalities and distinctions, and suggests directions for further investigation.

Submission Number: 37