Unsupervised Detailed Human Shape Estimation from Multi-view Color Images

Huayu Zheng, Kangkan Wang, Wei Li, Jian Yang

Abstract: This paper presents a novel framework to estimate detailed human body shape from color images in an unsupervised manner. It is a challenging task due to factors such as variations in human shapes, occlusion, and cloth details. The existing methods are mainly supervised and require a large number of ground truth real training data which is usually hard to obtain. To solve this problem, we propose an unsupervised detailed human shape estimation method from multi-view color images. Specifically, we first predict the depth map for the source view through robust photometric consistency with different views. Then, we predict the initial SMPL model from the color image and refine it by an iterative error feedback regressor based on point clouds of the predicted depth map. Finally, the refined SMPL model is deformed to fit the details (i.e., clothes and faces) on the point clouds to recover the detailed human shapes which are represented by adding a set of offsets to the SMPL model. The experimental results on different dataset demonstrate that our method outperforms the state-of-the-art methods and achieves higher reconstruction accuracy.