Capture and 3D Video Processing of Volumetric Video

Oliver Schreer, Ingo Feldmann, Sylvain Renault, Marcus Zepp, Markus Worchel, Peter Eisert, Peter Kauff

Abstract: Volumetric video is regarded worldwide as the next important development step in the field of media production. Especially in the context of the extremely rapid development of the Virtual Reality (VR) and Augmented Reality (AR) markets, volumetric video is becoming a key technology. In this paper, a new capture and processing system for volumetric video is presented, called 3D Human Body Reconstruction (3DHBR). The system is based on 16 stereo pairs of high-resolution cameras capturing a moving person in 360 degree. A novel stereo approach provides depth information from all perspectives, which is then fused to a single consistent 3D point cloud. A meshing and mesh reduction algorithm finally produces a sequence of meshes that can be integrated into common render engines. Given that, an integration of realistic dynamic 3D reconstructions of moving persons in VR and AR applications is possible.