Semi-automatic 2D-to-3D video conversion based on background sprite generation

Wen-Nung Lie, Shao-Ting Chiu, Jui-Chiu Chiang

Abstract: This paper presents a new technique for semi-automatic 2D to 3D stereo video conversion. Our algorithm escapes the scope of traditional depth propagation paradigm based on motion estimation and compensation. First of all, we treat the foreground and background depths separately and then combine them to form a final depth map for each video frame. For the foreground parts, they are first segmented out and given the region labels and depths for the two bounding key frames in a human interactive manner. The foreground region labels are then propagated to the intermediate non-key frames to construct the GMM models for foreground segmentation based on the well-known graph-cut algorithm. For the background parts, all video frames are integrated into a background sprite model (BSM), even for a moving camera, based on the image registration algorithm. Users can then draw detailed depth profiles for the integral BSM, thus reducing the human efforts required significantly. Experimental results show that our method is capable of retaining a more complete foreground contour and smooth background variation in depths than prior literature. This advantage is obviously helpful in view synthesis for 3D display and perception.