Dual Caption Preference Optimization for Diffusion Models

ACL ARR 2025 February Submission4641 Authors

Abstract:

Recent advancements in human preference optimization, originally developed for Large Language Models (LLMs), have shown significant potential in improving text-to-image diffusion models. These methods aim to learn the distribution of preferred samples while distinguishing them from less preferred ones. However, existing preference datasets often exhibit overlap between these distributions, leading to a *conflict distribution*.
Additionally, we identified that input prompts contain irrelevant information for less preferred images, limiting the denoising network's ability to accurately predict noise in preference optimization methods, known as the *irrelevant prompt* issue.
To address these challenges, we propose **Dual Caption Preference Optimization (DCPO)**, a novel approach that utilizes two distinct captions to mitigate irrelevant prompts. To tackle conflict distribution, we introduce the **Pick-Double Caption** dataset, a modified version of Pick-a-Pic v2 with separate captions for preferred and less preferred images. We further propose three different strategies for generating distinct captions: captioning, perturbation, and hybrid methods. Our experiments show that DCPO significantly improves image quality and relevance to prompts, outperforming other methods.

Paper Type: Long

Research Area: Generation

Research Area Keywords: analysis; data-to-text generation

Contribution Types: Publicly available software and/or pre-trained models, Data analysis, Theory

Languages Studied: English

Submission Number: 4641