Bidirectional Temporal Diffusion Model for Temporally Consistent Human Animation

Tserendorj Adiya, Jae Shin Yoon, JUNGEUN LEE, Sanghun Kim, Hwasup Lim

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Keywords: Human Motion Transfer, Diffusion Model, Human Animation, Generative model.

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.

Abstract:

We introduce a method to generate temporally coherent human animation from a single image, a video, or a random noise.
This problem has been formulated as modeling of an auto-regressive generation, i.e., to regress past frames to decode future frames.
However, such unidirectional generation is highly prone to motion drifting over time, generating unrealistic human animation with significant artifacts such as appearance distortion.
We claim that bidirectional temporal modeling enforces temporal coherence on a generative network by largely suppressing the appearance ambiguity.
To prove our claim, we design a novel human animation framework using a denoising diffusion model:
a neural network learns to generate the image of a person by denoising temporal Gaussian noises whose intermediate results are cross-conditioned bidirectionally between consecutive frames.
In the experiments, our method demonstrates strong performance compared to existing unidirectional approaches with realistic temporal coherence.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Primary Area: generative models

Submission Number: 4441