From Preferences to Prejudice: Alignment Tuning Amplifies Social Bias in Language-Conditioned Video Generation

Zefan Cai, Haoyi Qiu, Haozhe Zhao, Ke Wan, Jiachen Li, Jiuxiang Gu, Wen Xiao, Nanyun Peng, Junjie Hu

Keywords: Video Generaiton Evaluation, Social Bias

Abstract: Recent advances in video diffusion models have significantly enhanced text-to-video generation, particularly through \textit{alignment tuning} using reward models trained on human preferences. While these methods improve visual quality, they can unintentionally encode and amplify \textit{social biases}. To systematically trace how such biases evolve throughout the alignment pipeline, we introduce \benchmark, a comprehensive diagnostic framework for evaluating social representation in video generation. Grounded in established social bias taxonomies, \benchmark employs an \textit{event-based prompting} strategy to disentangle semantic content (verbs and contexts) from actor attributes (gender and ethnicity). It further introduces multi-granular metrics to evaluate (1) overall ethnicity bias, (2) gender bias conditioned on ethnicity, (3) distributional shifts in social attributes across model variants, and (4) the temporal persistence of bias within videos. Using this framework, we conduct the first end-to-end analysis connecting biases in \textit{human preference datasets}, their amplification in \textit{reward models}, and their propagation through \textit{alignment-tuned video diffusion models}. Our results reveal that alignment tuning not only strengthens representational biases but also makes them temporally stable, producing smoother yet more stereotyped portrayals. These findings highlight the need for bias-aware evaluation and mitigation throughout the alignment process to ensure fair and socially responsible video generation.

Email Sharing: We authorize the sharing of all author emails with Program Chairs.

Data Release: We authorize the release of our submission and author names to the public in the event of acceptance.

Submission Number: 133