Interactive Classification by Asking Informative Questions

Lili Yu, Howard Chen, Sida I. Wang, Yoav Artzi and Tao Lei,

TL;DR: We propose an interactive approach for classifying natural language queries by asking users for additional information using information gain and a reinforcement learning policy controller.

Abstract: We propose an interactive classification approach for natural language queries. Instead of classifying given the natural language query only, we ask the user for additional information using a sequence of binary and multiple-choice questions. At each turn, we use a policy controller to decide if to present a question or pro-vide the user the final answer, and select the best question to ask by maximizing the system information gain. Our formulation enables bootstrapping the system without any interaction data, instead relying on non-interactive crowdsourcing an-notation tasks. Our evaluation shows the interaction helps the system increase its accuracy and handle ambiguous queries, while our approach effectively balances the number of questions and the final accuracy.

Keywords: NLP, interactive classification, interactive system, text classification, data collection

Community Implementations: [ 1 code implementation](https://www.catalyzex.com/paper/arxiv:1911.03598/code)