Seeing Through Words: Controlling Visual Retrieval Quality with Language

ICLR 2026 Conference Submission3459 Authors

Keywords: Large Language Models, Vision-Language Models, Query Completion

Abstract: Text-to-image retrieval is a fundamental task in vision--language learning, yet in real-world scenarios it is often challenged by short and underspecified user queries. Such queries are typically only one or two words long, making them semantically ambiguous, prone to collisions across diverse visual interpretations, and lacking explicit control over the quality of retrieved images. To address these issues, we propose a new paradigm of quality-controllable retrieval, which enriches short queries with contextual details while incorporating explicit notions of image quality. Our key idea is to leverage a generative large language model as a query completion function, extending underspecified queries into descriptive forms that capture fine-grained visual attributes such as pose, scene, and aesthetics. We introduce a training framework that conditions query completion on discretized quality levels, derived from relevance and aesthetic scoring models, so that query enrichment is not only semantically meaningful but also quality-aware. The resulting system provides three key advantages: {1} flexibility, as it is compatible with any pretrained vision--language model without modification; {2} transparency, since enriched queries are explicitly interpretable by users; and {3} controllability, enabling retrieval results to be steered toward user-preferred quality levels. Extensive experiments demonstrate that our proposed approach significantly improves retrieval results and provides effective quality control, bridging the gap between the expressive capacity of modern vision--language models and the underspecified nature of short user queries.

Primary Area: applications to computer vision, audio, language, and other modalities

Submission Number: 3459