Go to ACL ARR 2025 February homepageIncorporating Discarded Candidate Tokens of Large Language Models for Efficient Query Expansion
In the era of Large Language Models (LLMs), efficient retrieval is crucial for integration with modern retrieval-augmented LLM systems, making sparse retrieval modules a popular choice due to their efficiency and robustness in low-resource settings. To enhance sparse retrieval performance, LLM-based Query Expansion (QE) has emerged as a solution to bridge the lexical gap between queries and documents. However, existing QE methods face a fundamental trade-off between efficiency and effectiveness, driven by the length of generated tokens. To address this, we propose Discarded candidate Tokens Query Expansion (DTQE), a novel query expansion method that leverages conventionally unselected tokens from the LLM’s decoding process by indexing them separately. Experimental results demonstrate that DTQE maintains high efficiency compared to more resource-intensive baselines while significantly outperforming keyword-based expansion ones.