Questions are Not All You Need for Brewing BeIR

Anonymous

TL;DR: We explore task adaptation with search intent.

Abstract: This paper studies the problem of information retrieval, to adapt to unseen tasks. Existing work generates synthetic queries from domain-specific documents to jointly train the retriever. However, the conventional query generator inadvertently or intentionally assumes the query as a question, thus failing to accommodate general search intents. A more lenient approach, which we refer to as retrieval with intent, incorporates task-specific elements into the query generation process, such as few-shot learning. In this paper, we explore novel strategies for task adaptation by guiding the LM to generate queries covering diverse search intents, using instructions and relevant demonstrations. We propose EGG, a query generator that better adapts to wide search intents expressed in the BeIR benchmark. Our method outperforms existing models on four tasks with underexplored intents, while utilizing 47 times smaller query generator compared to the previous state-of-the-art. Together, our work sheds light on how to integrate diverse search intents into the query generation process.

Paper Type: short

Research Area: Information Retrieval and Text Mining

Contribution Types: Approaches to low-resource settings

Languages Studied: English