Scalable 3D Object-centric Learning

Tianyu Wang, miaomiao Liu, Kee Siong Ng

Abstract:

We tackle the task of unsupervised 3D object-centric representation learning on scenes of potentially unbounded scale.
Existing approaches to object-centric representation learning exhibit significant limitations in achieving scalable inference due to their dependencies on a fixed global coordinate system.
In contrast, we propose to learn view-invariant 3D object representations in localized object coordinate systems.
To this end, we estimate the object pose and appearance representation separately and explicitly project object representations across views.
We adopt amortized variational inference to process sequential input and update object representations online.
To scale up our model to scenes with an arbitrary number of objects, we further introduce a Cognitive Map that allows the registration and querying of objects on a global map.
We employ the object-centric neural radiance field (NeRF) as our 3D scene representation, which is jointly inferred by our unsupervised object-centric learning framework.
Experimental results demonstrate that our method can infer and maintain object-centric representations of unbounded 3D scenes.
Further combined with a per-object NeRF finetuning process, our model can achieve scalable high-quality object-aware scene reconstruction.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics

Submission Guidelines: Yes

Please Choose The Closest Area That Your Submission Falls Into: Deep Learning and representational learning