Generating Long-Take Videos via Effective Keyframes and Guidance

Hsin-Ping Huang, Yu-Chuan Su, Ming-Hsuan Yang

Abstract: We tackle the challenge of generating long-take videos encompassing multiple non-repetitive yet coherent events. Existing approaches generate long videos conditioned on single input guidance, often leading to repetitive content. To address this problem, we develop a framework that uses multiple guidance sources to enhance long video generation. The main idea of our approach is to decouple video generation into keyframe generation and frame interpolation. In this process, keyframe generation focuses on cre-ating multiple coherent events, while the frame interpolation stage generates smooth intermediate frames between keyframes using existing video generation models. A novel mask attention module is further introduced to improve co-herence and efficiency. Experiments on challenging real-world videos demonstrate that the proposed method outper-forms prior methods by up to 9.5% in objective metrics.