Complex Motion-aware Splatting for Video Frame Interpolation

Minho Park, Yuseok Bae

Abstract: Video frame interpolation, a crucial component of computer vision, synthesizes additional frames to enhance the frame rate of a video, leading to improved performance with minimal additional cost. Despite recent advancements with deep learning and convolutional neural networks (CNNs), it still remains a challenge to generate precise intermediate frames, especially when complex and fast motions are involved. This paper presents a novel deep learning-based framework for video frame interpolation that incorporates a complex motion detection module and proposes a complex motion-aware splatting (CMS) method. We employ a forward warping approach that uses a complex motion map as a weight map in splatting. The framework further leverages a module that embeds temporal and spatial information from the frame sequence to acquire motion information. The effectiveness of our proposed model is demonstrated through qualitative and quantitative results on a public dataset.