PISCES: Annotation-free Text-to-Video Post-Training via Bi-objective OT-aligned Rewards

ICLR 2026 Conference Submission9580 Authors

Keywords: video generative models

TL;DR: PIESCES is the first annotation-free T2V post-training algorithm with a novel rewards module that outperforms both annotation-based and annotation-free T2V post-training methods.

Abstract: Text-to-video (T2V) generation aims to synthesize videos with high visual quality and temporal consistency that are semantically aligned with input text. Reward-based post-training has emerged as a promising direction to improve the quality and semantic alignment of generated videos. However, recent methods either rely on large-scale human preference annotations or operate on misaligned embeddings from pre-trained vision-language models, both leading to limited scalability or suboptimal supervision. We present $\texttt{PISCES}$, an annotation-free post-training algorithm that addresses these limitations via a novel Bi-objective Optimal Transport (OT)-aligned Rewards module. To align reward signals with human judgment, $\texttt{PISCES}$ uses OT to bridge text and video embeddings at both the distribution and discrete token levels, enabling reward supervision to fulfill two objectives: (i) a Distributional OT-aligned Quality Reward that captures overall visual quality and temporal coherence; and (ii) a Discrete Token-level OT-aligned Semantic Reward that enforces semantic, spatio-temporal correspondence between text and video tokens. To our knowledge, $\texttt{PISCES}$ is the first to improve annotation-free reward supervision in generative post-training through the lens of OT. Experiments on both short-video and long-video generation show that $\texttt{PISCES}$ outperforms both annotation-free and annotation-based methods on VBench across Quality and Semantic scores, with human preference studies further validating its effectiveness. We also show that the Bi-objective OT-aligned Rewards module is compatible with multiple optimization paradigms, including direct backpropagation and reinforcement learning fine-tuning.

Primary Area: generative models

Submission Number: 9580