Cascaded Network-based Single-View Bird 3D Reconstruction

Pei Su, Qijun Zhao

Abstract:

Existing single-view bird 3D reconstruction methods mostly cannot well recover the local geometry such as feet and wing tips, and the
resulting 3D models often appear to have poor appearance when viewed from a new perspective. We thus propose a new method that requires only images and their silhouettes to accurately predict the shape of birds, as well as to obtain reasonable appearance in new perspectives. The key to the method lies in the introduction of a cascaded structure in the shape reconstruction network. This allows for the gradual generation of the 3D shape of birds from coarse to fine, enabling better capturing of local geometric features. Meanwhile, we recover the texture, lighting and camera pose with attention-enhanced encoders. To further improve the plausibility of the reconstructed 3D bird in novel views, we introduce the Multi-view Cycle Consistency loss to train the proposed method. We compare our method with state-of-the-art methods and demonstrate its superiority both qualitatively and quantitatively.