How Should We Evaluate Data Deletion in Graph-Based ANN Indexes?

NeurIPS 2025 Workshop MLForSys Submission35 Authors

Keywords: Approximate Nearest Neighbor Search, ANNS, Deletion

TL;DR: We introduce a framework to evaluate data deletion in graph-based ANNS, formalizing three deletion methods and assessing their performance. Then, we propose Deletion Control, which dynamically selects deletion methods to maintain required accuracy.

Abstract:

Approximate Nearest Neighbor Search (ANNS) has recently gained significant attention due to its many applications, such as Retrieval-Augmented Generation.
Such applications require ANNS algorithms that support dynamic data, so the ANNS problem on dynamic data has attracted considerable interest.
However, a comprehensive evaluation methodology for data deletion in ANNS has yet to be established.
This study proposes an experimental framework and comprehensive evaluation metrics to assess the efficiency of data deletion for ANNS indexes under practical use cases.
Specifically, we categorize data deletion methods in graph-based ANNS into three approaches and formalize them mathematically.
The performance is assessed in terms of accuracy, query speed, and other relevant metrics.
Finally, we apply the proposed evaluation framework to Hierarchical Navigable Small World, one of the state-of-the-art ANNS methods, to analyze the effects of data deletion, and propose Deletion Control, a method which dynamically selects the appropriate deletion method under a required search accuracy.

Submission Number: 35