RewardSDS: Aligning Score Distillation via Reward- Weighted Sampling

TMLR Paper6181 Authors

Abstract: Score Distillation Sampling (SDS) has emerged as a highly effective technique for leveraging 2D diffusion priors for a diverse set of tasks such as text-to-3D generation. While powerful, SDS still struggles with achieving fine-grained alignment to user intent. To overcome this limitation, we introduce RewardSDS, a novel approach that weights noise samples based on the alignment scores of a reward model, producing a weighted SDS loss. This loss prioritizes gradients from noise samples that yield aligned high-reward output. Our approach is broadly applicable and can be applied to diverse methods extending SDS. In particular, we also demonstrate its applicability to Variational Score Distillation (VSD) by introducing RewardVSD. We evaluate RewardSDS on text-to-image, 2D editing, and text-to-3D generation tasks, demonstrating a significant improvement over SDS and subsequent baselines on a diverse set of metrics measuring generation quality and alignment to desired reward models.

Submission Length: Regular submission (no more than 12 pages of main content)

Assigned Action Editor: ~Dit-Yan_Yeung2

Submission Number: 6181