Task Planning for Visual Room Rearrangement under Partial Observability

Karan Mirakhor, Sourav Ghosh, Dipanjan Das, Brojeshwar Bhowmick

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Keywords: Task Planning, Object Search, Deep-RL, Robotics

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.

Abstract:

This paper presents a novel hierarchical task planner under partial observability
that empowers an embodied agent to use visual input to efficiently plan a sequence
of actions for simultaneous object search and rearrangement in an untidy room,
to achieve a desired tidy state. The paper introduces (i) a novel Search Network
that utilizes commonsense knowledge from large language models to find unseen
objects, (ii) a Deep RL network trained with proxy reward, along with (iii) a novel
graph-based state representation to produce a scalable and effective planner that
interleaves object search and rearrangement to minimize the number of steps taken
and overall traversal of the agent, as well as to resolve blocked goal and swap
cases, and (iv) a sample-efficient cluster-biased sampling for simultaneous training
of the proxy reward network along with the Deep RL network. Furthermore,
the paper presents new metrics and a benchmark dataset - RoPOR, to measure
the effectiveness of rearrangement planning. Experimental results show that our
method significantly outperforms the state-of-the-art rearrangement methods Weihs
et al. (2021a); Gadre et al. (2022); Sarch et al. (2022); Ghosh et al. (2022).

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Primary Area: applications to robotics, autonomy, planning

Submission Number: 1796