Improving Unsupervised Strict Zero-shot Intent Classification with Candidate Selection

ACL ARR 2025 February Submission2387 Authors

Abstract: Task-oriented dialogue systems allow users to interact through natural language with a variety of digital devices in order to accomplish some goal, within which intent classification is an integral component in ensuring the satisfaction of a user's request. Applications of Large Language Models (LLMs) in this domain can suffer from prohibitively high computation requirements and costs owing to the number of input tokens scaling with the number of intents. We propose a framework using candidate selection, aimed at refining a model's selection of candidate intents to reduce inference costs. We validate our approach through extensive evaluation on four commonly-used intent classification datasets and show that our candidate selection approach can improve zero-shot intent classification performance (between +2.08\% to +14.67\%) over naive zero-shot across a range of model parameters, while significantly reducing both the number of input tokens (up to 88\% reduction) and inference time (up to 53\% reduction). All the while accomplishing this without any additional fine-tuning.

Paper Type: Long

Research Area: Dialogue and Interactive Systems

Research Area Keywords: task-oriented,multilingual / low resource

Contribution Types: NLP engineering experiment, Approaches to low-resource settings, Approaches low compute settings-efficiency, Data analysis

Languages Studied: English

Submission Number: 2387