HERON: Human-robot collaboration with Efficient and Resilient OptimizatioN for Long-horizon planning

Taehyeon Kim, Gyeongmin Kim, E. Cho Smith, Byung-Cheol Min

Keywords: Human-Robot Collaboration, Long-Horizon Planning, Task Scheduling

TL;DR: A framework that combines LLM-based task decomposition, physics-guided estimation, and MILP optimization to enable efficient and resilient human–robot collaboration under uncertainty.

Abstract: The integration of humans into long-horizon planning introduces unique challenges that extend beyond conventional robotic task planning. Unlike robots, humans exhibit inherent uncertainty in task execution, including variable performance, unexpected interruptions, and dynamic goal changes, all of which complicate efficient collaboration. To address these challenges, we propose Human-robot collaboration with Efficient and Resilient OptimizatioN for Long-horizon planning (HERON), a novel framework that combines large language models (LLMs), physics-guided reasoning, and optimization techniques. HERON leverages LLMs in two complementary roles: (i) decomposing natural language task descriptions into structured sub-tasks with agent assignments, and (ii) generating physics-guided execution time estimates and determining sub-task assignments for both human and robot agents based on physical constraints and complementarities. These outputs are incorporated into a mixed-integer linear programming scheduler, which dynamically re-schedules based on observed human uncertainties. This integration ensures that scheduling is not only feasible with respect to physical limitations but also robust to human unpredictability while maintaining efficiency in resource and time allocation. Experiments demonstrate that HERON enables resilient and adaptive human-robot collaboration, achieving more efficient scheduling and higher task success rates compared to existing LLM-based planning frameworks. Website at https://sites.google.com/view/heron-planner.

Primary Area: applications to robotics, autonomy, planning

Submission Number: 9765