Learning Attribute-Aware Hash Codes for Fine-Grained Image Retrieval via Query Optimization

Peng Wang, Yong Li, Lin Zhao, Xiu-Shen Wei

Abstract: Fine-grained hashing has become a powerful solution for rapid and efficient image retrieval, particularly in scenarios requiring high discrimination between visually similar categories. To enable each hash bit to correspond to specific visual attributes, we propose a novel method that harnesses learnable queries for attribute-aware hash code learning. This method deploys a tailored set of queries to capture and represent nuanced attribute-level information within the hashing process, thereby enhancing both the interpretability and relevance of each hash bit. Building on this query-based optimization framework, we incorporate an auxiliary branch to help alleviate the challenges of complex landscape optimization often encountered with low-bit hash codes. This auxiliary branch models high-order attribute interactions, reinforcing the robustness and specificity of the generated hash codes. Experimental results on benchmark datasets demonstrate that our method generates attribute-aware hash codes and consistently outperforms state-of-the-art techniques in retrieval accuracy and robustness, especially for low-bit hash codes, underscoring its potential in fine-grained image hashing tasks.

Lay Summary:

With the explosive growth of fine-grained data in real applications, fine-grained image retrieval has a wide range of application scenarios. Fine-grained hashing, which maps images to compact binary codes, has emerged as a promising solution for large-scale retrieval tasks, as it significantly reduces storage costs and improves retrieval speed.

Our paper proposes an effective solution. We model the hashing problem as a set prediction task, where each bit of the generated hash codes indicates whether the image possesses a specific visual attribute. Additionally, we analyze the limitations caused by a large number of categories and small feature dimensions in low-bit scenarios. Based on our analysis, we seamlessly incorporate an auxiliary branch with a query learning mechanism, which significantly improves the model's performance.

Extensive experimental results demonstrate that our attribute-aware hashing method not only achieves excellent retrieval performance but also shows that each bit of the hash codes is interpretable.

Link To Code: https://github.com/SEU-VIPGroup/QueryOpt

Primary Area: Applications->Computer Vision

Keywords: Large-Scale Fine-Grained Image Retrieval, Learning-to-Hash, Attribute-Aware, Deep Learning

Submission Number: 8707