The impact of adding perspective-taking to spatial referencing during human-robot interaction

Fethiye Irmak Dogan, Sarah Gillet, Elizabeth J. Carter, Iolanda Leite

Abstract: Highlights • Robots can dynamically generate spatial referring expressions from a user’s perspective. • Referring to objects from users’ perspectives helps people perform the task faster. • Perspective-taking facilitates accurate task completion. • The task is perceived as easier when objects are described from users’ perspectives. • Perspective-taking improves task efficiency in human–robot interaction. Abstract For effective verbal communication in collaborative tasks, robots need to account for the different perspectives of their human partners when referring to objects in a shared space. For example, when a robot helps its partner find correct pieces while assembling furniture, it needs to understand how its collaborator perceives the world and refer to objects accordingly. In this work, we propose a method to endow robots with perspective-taking abilities while spatially referring to objects. To examine the impact of our proposed method, we report the results of a user study showing that when the objects are spatially described from the users’ perspectives, participants take less time to find the referred objects, find the correct objects more often and consider the task easier.