RealDPO: Real or Not Real, that is the Preference

ICLR 2026 Conference Submission15200 Authors

Keywords: Video Generation, Motion Synthesis, Direct Preference Optimization (DPO), Preference Learning, Alignment Paradigm

TL;DR: RealDPO is a new alignment method that uses real-world videos as the win samples in Direct Preference Optimization (DPO) to significantly improve the realism of motions generated by video generative models.

Abstract: Video generative models have recently achieved notable advancements in synthesis quality. However, generating complex motions remains a critical challenge, as existing models often struggle to produce natural, smooth, and contextually consistent movements. This gap between generated and real-world motions limits their practical applicability. To address this issue, we introduce RealDPO, a novel alignment paradigm that leverages real-world data as positive samples for preference learning, enabling more accurate motion synthesis. Unlike traditional supervised fine-tuning (SFT), which offers limited corrective feedback, RealDPO employs Direct Preference Optimization (DPO) with a tailored loss function to enhance motion realism. By contrasting real-world videos with erroneous model outputs, RealDPO enables iterative self-correction, progressively refining motion quality. To support post-training in complex motion synthesis, we propose RealAction-5K, a curated dataset of high-quality videos capturing human daily activities with rich and precise motion details. Extensive experiments demonstrate that RealDPO significantly improves video quality, text alignment, and motion realism compared to state-of-the-art models and existing preference optimization techniques.

Primary Area: generative models

Submission Number: 15200