ECLM: Entity Level Language Model for Spoken Language Understanding with Chain of Intent
Abstract: Large Language Models (LLMs) have shown remarkable success in language generation, demonstrating broad competence across different tasks. However, their direct application to spoken language understanding (SLU) remains challenging. This is particularly true for token-level tasks, where the autoregressive architecture of LLMs can lead to error propagation and misalignment problems.
In this paper, we present the Entity-level Language Model (ECLM) framework for SLU, which addresses these challenges by transforming the traditional token-level slot-filling task into an entity recognition problem. In addition, we propose a novel concept, "Chain of Intent", which enables LLMs to effectively handle multi-intent recognition in a step-by-step manner.
Our experiments demonstrate that ECLM achieves substantial improvements over state-of-the-art pre-trained models like Uni-MIS, with overall accuracy gains of 3.7\% on the MixATIS dataset and 3.1\% on the MixSNIPS dataset. Moreover, the ECLM framework surpasses conventional supervised fine-tuning of LLMs, delivering improvements of 8.5\% and 21.2\% on MixATIS and MixSNIPS, respectively.
Paper Type: Long
Research Area: Speech Recognition, Text-to-Speech and Spoken Language Understanding
Research Area Keywords: Intent detection,Slot Filling
Contribution Types: Model analysis & interpretability, NLP engineering experiment, Approaches to low-resource settings
Languages Studied: English
Submission Number: 563
Loading