IRGen: Generative Modeling for Image Retrieval
Primary Area: generative models
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Keywords: Image retrieval, autoregressive model, generative model
Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.
TL;DR: We make first attempt to adapt generative modeling to image retrieval and demonstrate its effectiveness by showing significant improvement over widely used benchmarks.
Abstract: While generative modeling has become prevalent across numerous research fields, its potential application to image retrieval has yet to be thoroughly justified. In this paper, we present a novel approach, reframing image retrieval as a variant of generative modeling and employing a sequence-to-sequence model. This provides promising alignment with the overarching theme of unification in current research. Our framework enables end-to-end differentiable search, leading to superior performance through direct optimization. During the development of IRGen, we tackle the key technical challenge of converting an image into a concise sequence of semantic units, which is essential to facilitate efficient and effective search. Extensive experiments demonstrate that our model yields significant improvement over various widely utilized benchmarks, and further validate its performance on million-scale datasets. Besides, the substantial enhancement of precision scores achieved through generative modeling, potentially opens the avenue to excluding the rerank stage typically utilized in practical retrieval pipelines.
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Submission Number: 2547
Loading