Multiview texture coding and free viewpoint image synthesis for mesh-based 3D video transmission

Jui-Chiu Chiang, Ping-He Hou, Kai-Che Liu, Wen-Nung Lie

Abstract: In this paper, the “3D mesh model plus texture” strategy is adopted to implement a 3D video system. First, the acquired multi-view videos are processed to construct dynamic 3D mesh models about the foreground subject. These 3D mesh models, together with the acquired texture information, are then compressed and transmitted via networks to receivers. Main contributions of our work lies on the proposals of data reduction and pre-processing of multi-view texture information before H.264/AVC encoding at transmitter and a robust occlusion test on synthesizing novel views at receiver. Our proposed data reduction and pre-processing schemes are capable of removing redundant texture information, while maintaining inter-frame correlation, to result in high coding efficiency. Experiment results show that the proposed occlusion test is capable of eliminating texture-rendering artifacts due to 3D model reconstruction errors, thus improving the viewing quality of 3D video at receiver. Besides, our texture encoder achieves a saving of 40% ~ 57% in transmission bit rate, compared with the traditional approach.