DanceU: motion-and-music-based automatic effect generation for dance videos

Yanjie Pan, Yaru Du, Shandong Wang, Yun Ye, Yong Jiang, Zhen Zhou, Li Xu, Ming Lu, Yunbiao Lin, Jiehui Lu

Abstract: We propose a novel method, named DanceU, to automate dance video editing in an intelligent way. Our method fully leverages two folds of a dance video, the background music and the dancer’s motion, to determine where to add effects and what kind of effects to be added. We parameterize dance motion into two metrics: motion amplitude and motion intensity, to represent the motion characteristics of different dance types. Then, we design algorithms to align the dance motions with music beats and automatically decide dance effects for the dance video. To our best knowledge it is the first proposed method for automatic dance video effect generation. Results show that our method can generate fluent, performative, and music-matched effects, which achieves similar editing level to professionals. (A demo video: https://f.io/fgQUxehZ)