Learning to Anneal and Prune Proximity Graphs for Similarity Search

Minjia Zhang, Wenhan Wang, Yuxiong He

Keywords: Similarity search, Proximity graph, Learning to prune, Edge heterogeneity, Annealing, Efficiency

TL;DR: Annealable proximity graphs facilitates similarity search by learning to prune inferior edges without drastically changing graph properties.

Abstract: This paper studies similarity search, which is a crucial enabler of many feature vector--based applications. The problem of similarity search has been extensively studied in the machine learning community. Recent advances of proximity graphs have achieved outstanding performance through exploiting the navigability of the underlying graph structure. In this work, we introduce the annealable proximity graph (APG) method to learn and reshape proximity graphs for efficiency and effective similarity search. APG makes proximity graph edges annealable, which can be effectively trained with a stochastic optimization algorithm. APG identifies important edges that best preserve graph navigability and prune inferior edges without drastically changing graph properties. Experimental results show that APG achieves state-of-the-art results not only by producing proximity graphs with less number of edges but also speeding up the search time by 20--40\% across different datasets with almost no loss of accuracy.

Code: https://drive.google.com/open?id=15vGhNS0O9l-zPAbdPAxwxIzV558jjGeQ