Gradient-Based Diversity Optimization with Differentiable Top-$k$ Objective

ICLR 2026 Conference Submission25123 Authors

Keywords: Diversity Optimization, Gradient-based learning, Recommendation

TL;DR: We introduce a differentiable top-k diversity objective with direct and indirect optimization, showing fine-tuning quickly adds diversity at scale with negligible accuracy loss.

Abstract: Predicting relevance is a pervasive problem across digital platforms, covering social media, entertainment, and commerce. However, when optimized solely for relevance and engagement, many machine-learning models amplify data biases and produce homogeneous outputs, reinforcing filter bubbles and content uniformity. To address this issue, we introduce a pairwise top-k diversity objective with a differentiable smooth-ranking approximation, providing a model-agnostic way to incorporate diversity optimization directly into standard gradient-based learning. Building on this objective, we cast relevance and diversity as a joint optimization problem, we analyze the resulting gradient trade-offs, and propose two complementary strategies: direct optimization, which modifies the learning objective, and indirect optimization, which reweights training data. Both strategies can be applied either when training models from scratch or when fine-tuning existing relevance-optimized models. We use recommendation as a natural evaluation setting where scalability and diversity are critical, and show through extensive experiments that our methods consistently improve diversity with negligible accuracy loss. Notably, fine-tuning with our objective is especially efficient, requiring only a few gradient steps to encode diversity at scale.

Primary Area: other topics in machine learning (i.e., none of the above)

Submission Number: 25123