Symmetry-based Learning of Radiance Fields for Rigid Objects

Zhiwei Han, Stefan Matthes, Hao Shen, Yuanting Liu

Submission Track: Extended Abstract

Keywords: 3D Computer Vision; Group Symmetry; Object-centric Learning

TL;DR: In this work, we present SymObjectRF, a symmetry-based method that learns 3D representations of rigid objects from dynamic scenes without hand-crafted annotation.

Abstract: In this work, we present SymObjectRF, a symmetry-based method that learns object-centric representations for rigid objects from one dynamic scene without hand-crafted annotations. SymObjectRF learns the appearance and surface geometry of all dynamic object in their canonical poses and represents individual object within its canonical pose using a canonical object field (COF). SymObjectRF imposes group equivariance on rendering pipeline by transforming 3D point samples from world coordinate to object canonical poses. Subsequently, a permutation-invariant compositional renderer combines the color and density values queried from the learned COFs and reconstructs the input scene via volume rendering. SymObjectRF is then optimized by minimizing scene reconstruction loss. We show the feasibility of SymObjectRF in learning object-centric representations both theoretically and empirically.

Submission Number: 11