Moving Out: Physically-grounded Human-AI Collaboration

ICLR 2026 Conference Submission14582 Authors

Keywords: Human-AI Collaboration, Physically-grounded

TL;DR: We introduce Moving Out and BASS for stronger human-AI collaboration under physical constraints.

Abstract:

The ability to adapt to physical actions and constraints in an environment is crucial for embodied agents (e.g., robots) to effectively collaborate with humans.
Such physically grounded human-AI collaboration must account for the increased complexity of the continuous state-action space and constrained dynamics caused by physical constraints.
In this paper, we introduce Moving Out, a new human-AI collaboration benchmark that resembles a wide range of collaboration modes affected by physical attributes and constraints, such as moving heavy items together and maintaining consistent actions to move a big item around a corner.
Using Moving Out, we designed two tasks and collected human-human interaction data to evaluate models' abilities to adapt to diverse human behaviors and unseen physical attributes.
To address the challenges in physical environments, we propose a novel method, BASS (Behavior Augmentation, Simulation, and Selection), to enhance the diversity of agents and their understanding of the outcome of actions.
Our experiments show that BASS outperforms state-of-the-art models in AI-AI and human-AI collaboration.

Primary Area: reinforcement learning

Submission Number: 14582