Abstract: Hierarchical Navigable Small World (HNSW) graph is used for approximate nearest neighbor (ANN). However, HNSW's fixed parameters can degrade performance when user query distributions misalign with training data. In this work, we propose a Deep Q-Network (DQN)-based framework named Adaptive Deep Q-Network (ADQN) to overcome this problem. ADQN models HNSW optimization as a Markov Decision Process, enabling adaptation to discrepancies between user query and training data distributions without requiring re-indexing. We evaluated our system on DBpedia, IGB and MS Marco datasets with adjustments of changing user preferences and adaptive query embeddings, and results show that our ADQN method achieves superior accuracy-latency trade-offs, lower maintenance overhead, and remains robust under evolving conditions, which can improve the performance of ANN through adaptive search.
Paper Type: Long
Research Area: Machine Learning for NLP
Research Area Keywords: Information Retrieval, Reinforcement Learning, Parameter-efficient-training, Data-efficient training, Retrieval-augmented generation
Contribution Types: Model analysis & interpretability, NLP engineering experiment, Approaches low compute settings-efficiency, Theory
Languages Studied: English in general
Submission Number: 2309
Loading