The State of Intent Detection in the Era of Large Autoregressive Language Models

Anonymous

Abstract: In-context learning (ICL) using large pre-trained autoregressive language models (LLMs, e.g. GPT-3) has demonstrated effective classification performance at a variety of natural language tasks. Using LLMs for intent detection is challenging due to the large label space and limited context window, such that it is difficult to fit a sufficient number of examples in the prompt to allow the use of in-context learning. In this paper, dense retrieval is used to bypass this limitation, giving the model only a partial view of the full label space. We show that retriever-augmented large language models are an effective way to tackle intent detection, bypassing context window limitations effectively through the retrieval mechanism. Comparing the LLaMA and OPT model families at different scales, we set new state of the art performance in the few-shot setting with zero training for two of the three intent classification datasets that we consider, while achieving competitive results on the third one. This work demonstrates that the Retriever+ICL framework is a strong zero-training competitor to fine-tuned intent detection approaches. In addition, a small study on the number of examples provided at different model scales is done, showing that larger models are needed to make effective use of more examples in-prompt.

Paper Type: short

Research Area: Dialogue and Interactive Systems