Towards Finer Human Reconstruction for Single RGB-D Images

Yan Zhu, Yu Qian, Renlong Dai, Linbo Wang, Zhengyi Liu, Xianyong Fang

Abstract: Existing methods on the parametric model assisted human surface reconstruction from single RGB-D images are still difficult to obtain fine results. This article proposes an improved method which includes three tactics to overcome this limitation. First, a direct optimization scheme is adopted to refine the parametric model for better back prior, considering that the estimated model can be inaccurate and thus affect the reconstruction performances. Second, a new encoder-decoder structured residual-feature based back refinement network is proposed to further polish the initial back surface. It can preserve the global human shapes and poses without missing body parts while keeping local details. Here, a learnable weighted based cross attention module (LCA) is embedded, which adaptively merges the residual features in high levels from both the SMPL-X and initial back depths via cross-attention for rich details. Thirdly, a new silhouette loss on both front and back surfaces is introduced, so that fine back surfaces with smooth transition between the front and back can be reached. With those three tactics, a novel framework is proposed for robust surface reconstruction for single RGB-D images. Experiment results show that the proposed approach can obtain surfaces with significant details without missing parts.

External IDs:dblp:conf/cgi/ZhuQDWLF24