Text2Interaction: Establishing Safe and Preferable Human-Robot Interaction

Jakob Thumm, Christopher Agia, Marco Pavone, Matthias Althoff

Keywords: Human-Robot Interaction, Human Preference Learning, Task and Motion Planning, Safe Control

TL;DR: Text2Interaction integrates human preferences into the task, motion, and control level of the robotic stack.

Abstract: Adjusting robot behavior to human preferences can require intensive human feedback, preventing quick adaptation to new users and changing circumstances. Moreover, current approaches typically treat user preferences as a reward, which requires a manual balance between task success and user satisfaction. To integrate new user preferences in a zero-shot manner, our proposed Text2Interaction framework invokes large language models to generate a task plan, motion preferences as Python code, and parameters of a safety controller. By maximizing the combined probability of task completion and user satisfaction instead of a weighted sum of rewards, we can reliably find plans that fulfill both requirements. We find that 83% of users working with Text2Interaction agree that it integrates their preferences into the plan of the robot, and 94% prefer Text2Interaction over the baseline. Our ablation study shows that Text2Interaction aligns better with unseen preferences than other baselines while maintaining a high success rate. Real-world demonstrations and code are made available at [sites.google.com/view/text2interaction](sites.google.com/view/text2interaction).

Video: https://www.youtube.com/watch?v=LNfVw9ZtAtI

Website: https://sites.google.com/view/text2interaction/

Code: https://github.com/JakobThumm/text2interaction

Student Paper: yes

Submission Number: 577