Ask the Right Questions: Active Question Reformulation with Reinforcement LearningDownload PDF

15 Feb 2018 (modified: 22 Oct 2023)ICLR 2018 Conference Blind SubmissionReaders: Everyone
Abstract: We frame Question Answering (QA) as a Reinforcement Learning task, an approach that we call Active Question Answering. We propose an agent that sits between the user and a black box QA system and learns to reformulate questions to elicit the best possible answers. The agent probes the system with, potentially many, natural language reformulations of an initial question and aggregates the returned evidence to yield the best answer. The reformulation system is trained end-to-end to maximize answer quality using policy gradient. We evaluate on SearchQA, a dataset of complex questions extracted from Jeopardy!. The agent outperforms a state-of-the-art base model, playing the role of the environment, and other benchmarks. We also analyze the language that the agent has learned while interacting with the question answering system. We find that successful question reformulations look quite different from natural language paraphrases. The agent is able to discover non-trivial reformulation strategies that resemble classic information retrieval techniques such as term re-weighting (tf-idf) and stemming.
TL;DR: We propose an agent that sits between the user and a black box question-answering system and which learns to reformulate questions to elicit the best possible answers
Keywords: machine translation, paraphrasing, question answering, reinforcement learning, agents
Code: [![Papers with Code](/images/pwc_icon.svg) 2 community implementations](https://paperswithcode.com/paper/?openreview=S1CChZ-CZ)
Data: [COCO](https://paperswithcode.com/dataset/coco), [Paralex](https://paperswithcode.com/dataset/paralex), [SQuAD](https://paperswithcode.com/dataset/squad), [SearchQA](https://paperswithcode.com/dataset/searchqa)
Community Implementations: [![CatalyzeX](/images/catalyzex_icon.svg) 2 code implementations](https://www.catalyzex.com/paper/arxiv:1705.07830/code)
9 Replies

Loading