Unsupervised Learning of Object-Centric Representation from Multi-Viewpoint Scenes

Tonglin Chen, Yinxuan Huang, Zhimeng Shen, Jinghao Huang, Lin Zhu, Dakun Yang, Bin Li, Xiangyang Xue

Keywords: Object-Centric Learning, Mulit-Viewpoints Learning

Abstract: Objects in a 2D image are influenced by factors like perspective, illumination, and occlusion in the corresponding 3D scene. This results in the challenge of identifying objects across different viewpoints. Humans can effortlessly identify objects from different viewpoints by recognizing their invariant characteristics in 3D dimensions. Motivated by this observation, we propose an object-centric learning method named Learning Object-centric Representation from Multi-viewpoint (LORM), which learns the representations of objects from multi-viewpoint scenes without any supervision. LORM leverages a novel slot attention encoder to decompose the representation of a scene into two distinct components: a viewpoint representation and several object representations. The former encompasses the viewpoint-dependent attributes (i.e., camera position and lighting) of the image observed from each viewpoint, while the latter captures the viewpoint-independent features (i.e., appearance, shape, scale, rotation and position) of the object across various perspectives. We propose a mixture patch decoder to enable LORM to simultaneously handle complex scenes and reconstruct an individual object's 2D appearance and shape at a specific viewpoint through the corresponding object representation and viewpoint representation. Extensive experiments are conducted on three complex simulation datasets, and the results demonstrate that our proposed method outperforms compared methods in individual object reconstruction while achieving comparable performance in scene decomposition.

Primary Area: unsupervised, self-supervised, semi-supervised, and supervised representation learning

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 7124