Learning and Evaluating Visual Similarity Discovery under Incomplete Labeling

ICLR 2026 Conference Submission19137 Authors

Keywords: visual similarity discovery

Abstract: Visual Similarity Discovery (VSD) focuses on retrieving positives: images of distinct objects that exhibit perceptual similarity to a given query. This is a core need in applications like e-commerce and visual search. This work advances VSD research through several key contributions. First, we introduce a new VSD dataset in the furniture domain with over 63K labeled image pairs, providing a valuable resource for VSD learning and evaluation. Second, we propose two evaluation metrics that enable more reliable and consistent VSD performance assessment under incomplete labeling. Third, we show that supervised finetuning of multiple pretrained models on VSD labels significantly improves VSD performance. Moreover, we present Soft Positive Augmentation, a method that leverages existing VSD labels to infer soft positive relations among unlabeled pairs via weighted graph transitivity. Augmenting the VSD labels with these inferred soft positives during finetuning yields additional performance gains. Our code and dataset will be made publicly available.

Primary Area: datasets and benchmarks

Submission Number: 19137