Monocular Real-Time Human Geometry Reconstruction

Qiao Feng, Yebin Liu, Yu-Kun Lai, Jing-Yu Yang, Kun Li

Abstract: Monocular human reconstruction is very popular in computer vision and graphics, which can be a backbone for various applications, such as 3DTV and Holographic Telepresence systems. In this demo, we present the first 30+FPS monocular real-time human geometry reconstruction system, which achieves high-quality results and real-time speed at the same time. Our system reconstructs a 3D human mesh for each frame with Fourier Occupancy Field (FOF), which is a very efficient representation for monocular reconstruction. We adapt it to support the perspective camera setup and build a concise reconstruction framework. HDRis and ray tracing rendering are used to improve the results on the real captured images. Furthermore, the newly designed framework can ignore background automatically, eliminating the need for segmentation, which reduces the usage of GPU and enables more post-processing. Our system can be deployed on PCs with high-performance GPUs and common USB web cameras, which is a consumer-accessible solution for human geometry reconstruction.