Fast Intent Classification for LLM Routing via Statistical Analysis of Representations

ICLR 2026 Conference Submission13933 Authors

Keywords: Intent Classification, Large Language Models, Statistical Analysis

Abstract: Intent classification in Large Language Models (LLMs) involves categorizing user prompts into predefined classes. For instance, given a user prompt, the system must determine whether it primarily concerns mathematics, coding, or general text processing. Such classification enables routing prompts to specialized models optimized for specific domains, improving both accuracy and computational efficiency. In this work, we introduce two lightweight, training-free methods based on statistical analysis of internal model representations and systematically compare them against baseline training-based approaches from the literature. Our methods analyze the distribution of key statistical metrics extracted from hidden features, enabling intent inference during the initial forward pass with minimal computational overhead. Through comprehensive empirical evaluation, we demonstrate that our training-free methods successfully classify prompts across varying levels of granularity—from high level distinctions (mathematics vs. coding vs. natural language) to fine-grained ones (e.g. Java vs. Python, etc). Our results provide a systematic characterization of scenarios where training-free methods are most useful, and identify cases where training-based approaches remain necessary, offering a practical guidance for deployment in production LLM systems.

Primary Area: foundation or frontier models, including LLMs

Submission Number: 13933