$\alpha$-Reachable Graphs for Multivector Nearest Neighbor Search

Siddharth Gollapudi, Ravishankar Krishnaswamy, Sandeep Silwal, Kirankumar Shiragur, Harsh Wardhan, Ben Landrum, Nikhil Rao

Keywords: Nearest neighbor Search, Graph-based Search, Multivector Retrieval

TL;DR: We prove upper bounds and demonstrate empirical performance on graphs with an asymmetric multi-vector distance function.

Abstract:

It is common in machine learning pipelines to embed every data point as an embedding in order to geometrically represent semantic relationships. However, the standard practice of using a single vector per data point may not be powerful enough to capture nuanced information in modalities such as text. Indeed, recent empirical evidence, such as the seminal ColBERT paper \citep{khattab2020colbert}, demonstrates the advantage of representing every data point as a collection of embeddings, and comparing data points via a \emph{multi-vector} similarity measure between collections of embeddings.

To accelerate the adoption of the multi-filter approach in large scale search and retrieval applications, efficient algorithms for nearest neighbor search for multi-vector similarities are needed. While many recent practical solutions have been proposed towards this problem, they are either limited to specific multi-vector similarities (such as the Chamfer distance used in ColBERT) or come with limited theoretical understanding. Our work aims to address this gap.

- On the theoretical side, we provide a provably efficient algorithm for approximate nearest neighbor search for a wide range of multi-vector similarities, including the Chamfer distance.
- Practically, we demonstrate that our approach can provide improved results for the common case of Chamfer similarity studied in prior empirical works. Our algorithm outperforms prior SOTA by up to **20\%** increase in QPS for comparable 1@100 recall, while achieving up to **2x** improvement for the challenging 100@100 recall setting.

The core of our approach lies in extending the well-known single-vector DiskANN algorithm, both in theory and practice, to the multi-vector setting in a black-box manner.

Primary Area: other topics in machine learning (i.e., none of the above)

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.

Reciprocal Reviewing: I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 10185