Motion Score Matching in Video Generation

Jiajun Zha, Ser-Nam Lim, Harry Yang

Keywords: Diffusion, Score Matching, Video Generation

TL;DR: Explicit score matching for motions in video generation to enhance the overall realism.

Abstract:

Subtle motions in videos are critical for overall realism for video generation but are overlooked by current score distillation methods like distribution matching distillation. Current distillation methods prefer to match the style first, %because that will
since it takes up most of the numerical significance. Such a distillation scheme will only create poorly generated motions, severely degrading the overall realism after distillation.
To address this, we propose motion score and enforce the matching of motion distribution in distillation.
We show that matching motion distribution is vital for the quality of generated videos.

Primary Area: generative models

Submission Number: 7701