Negative-Guided Subject Fidelity Optimization for Zero-Shot Subject-Driven Generation

ICLR 2026 Conference Submission19415 Authors

Keywords: text-to-image, subject-driven-tti

TL;DR: We propose the comparison-based fine-tuning strategy with synthetic negative data for improving subject fidelity in zero-shot subject-driven TTI.

Abstract: We present Subject Fidelity Optimization (SFO), a novel comparative learning framework for zero-shot subject-driven generation that enhances subject fidelity. Existing supervised fine-uning methods, which rely only on positive targets and use the diffusion loss as in the pre-training stage, often fail to capture fine-grained subject details. To address this, SFO introduces additional synthetic negative targets and explicitly guides the model to favor positives over negatives through pairwise comparison. For negative targets, we propose Condition-Degradation Negative Sampling (CDNS), which automatically produces synthetic negatives tailored for subject-driven generation by introducing controlled degradations that emphasize subject fidelity and text alignment without expensive human annotations. Moreover, we reweight the diffusion timesteps to focus fine-tuning on intermediate steps where subject details emerge. Extensive experiments demonstrate that SFO with CDNS significantly outperforms recent strong baselines in terms of both subject fidelity and text alignment on a subject-driven generation benchmark.

Primary Area: generative models

Submission Number: 19415