Fast Intent Classification for LLM Routing via Statistical Analysis of Representations

Nan Chen, Zhouhao Yang, Soufiane Hayou

Keywords: Intent Classification, Large Language Models, Statistical Analysis

Abstract:

Intent classification in Large Language Models (LLMs) involves categorizing user prompts into predefined classes. For instance, given a user prompt, the system must determine whether it primarily concerns mathematics, coding, or general text processing. Such classification enables routing prompts to specialized models optimized for specific domains, improving both accuracy and computational efficiency.
In this work, we conduct a systematic study comparing training-free vs training-based approaches for intent classification. For this purpose, we introduce two lightweight, training-free methods based on statistical analysis of internal model representations and compare them against MLP classifiers and linear probes. The training-free methods use key statistical metrics from hidden features, enabling intent inference during the initial forward pass with minimal overhead. Our comprehensive empirical evaluation reveals that 1) both training-free and training-based methods saturate easy benchmarks (mathematics vs. coding vs. natural language), 2) Training-based classifiers have an advantage on harder classification tasks (e.g. Java vs Python), and 3) Training-free methods are more robust to out-of-distribution and ambiguous prompts.

Primary Area: foundation or frontier models, including LLMs

Submission Number: 13933