Every Mistake Counts: Spatial and Temporal Beliefs for Mistake Detection in Assembly Tasks

Guodong Ding, Fadime Sener, Shugao Ma, Angela Yao

Primary Area: applications to robotics, autonomy, planning

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Keywords: mistake detection

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.

Abstract:

Assembly tasks, as an integral part of daily routines and activities, involve a series of sequential steps that are prone to error. This paper proposes a novel method for identifying ordering mistakes in assembly tasks based on knowledge-grounded beliefs. The beliefs comprise spatial and temporal aspects, each serving a unique role.
Spatial beliefs capture the structural relationships among assembly components, indicating their topological feasibility. Temporal beliefs model the action preconditions and enforce sequencing constraints. Furthermore, we introduce a learning algorithm that dynamically updates and augments the belief sets in an online manner.
To evaluate our approach, we first test its ability to deduce the predefined rules using synthetic data from industry assembly, and then apply it to a real-world dataset, enhanced with a new collection of annotations providing part information. We demonstrate our framework achieves superior performance in detecting ordering mistakes under both synthetic and real-world settings.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 3060