CoSeC-LCD: Controllable Self-Contrastive Latent Consistency Distillation for Better and Faster Human Animation Generation
Keywords: Consistency Distillation; Controllable and Consistent Generation
Abstract: Generating pose-driven and reference-consistent human animation has significant practical applications, yet it remains a prominent research challenge, facing substantial obstacles. A major issue with widely adopted diffusion-based methods is their slow generation speed, which is primarily due to multi-step iterative denoising processes. To tackle this challenge, we take the pioneering step of proposing the ReferenceLCM architecture, which utilizes latent consistency models (LCM) to facilitate accelerated generation. Additionally, to address hallucinations in fine-grained control, we introduce the Controllable Self-Contrastive Latent Consistency Distillation (CoSeC-LCD) regularization method. Our approach introduces a novel perspective by categorizing tasks into various classes and employing contrastive learning to capture underlying patterns. Building on this insight, we implement a hierarchical optimization strategy that significantly enhances animation quality across both spatial and temporal aspects. Comprehensive qualitative and quantitative experiments reveal that our method achieves results comparable to, or even surpassing, many state-of-the-art approaches, enabling high-fidelity human animation generation within just 2-4 inference steps.
Primary Area: applications to computer vision, audio, language, and other modalities
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
Reciprocal Reviewing: I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Submission Number: 1476
Loading