Imagined Subgoals for Hierarchical Goal-Conditioned Policies

Xuhui Kang, Wenqian Ye, Yen-Ling Kuo

Keywords: Imagined Subgoals, Hierarchical Planning, Goal-Conditioned Policy

Abstract: Humans form mental images of tasks being performed and their expected outcomes to support how they act in the world. We present a novel framework that generates visual subgoals based on the goal image and the robot's current state. The generated visual subgoals are used to guide the rollout of the policy. The subgoal generator is jointly trained with a progress encoder to track the progress of a task while constructing the next visual subgoals. At inference time, the robot adaptively samples a new visual subgoal from the generator once the current subgoal is achieved, or if the maximum step limit is reached without achieving the subgoal. We train our models and validate the proposed framework using the CALVIN manipulation dataset, demonstrating the improved success rate in various tasks.

Submission Number: 22