Syntax Matters: Towards Spoken Language Understanding via Syntax-Aware Attention

Yifeng Xie, Zhihong Zhu, Xuxin Cheng, Zhiqi Huang, Dongsheng Chen

Submission Type: Regular Short Paper

Submission Track: Syntax, Parsing and their Applications

Keywords: Spoken Language understanding, Syntactic Dependency Parsing, Feature Fusion

Abstract: Spoken Language Understanding (SLU), a crucial component of task-oriented dialogue systems, has consistently garnered attention from both academic and industrial communities. Although incorporating syntactic information into models has the potential to enhance the comprehension of user utterances and yield impressive results, its application in SLU systems remains largely unexplored. In this paper, we propose a carefully designed model termed Syntax-aware attention (SAT) to enhance SLU, where attention scopes are constrained based on relationships within the syntactic structure. Experimental results on three datasets show that our model achieves substantial improvements and excellent performance. Moreover, SAT can be integrated into other BERT-based language models to further boost their performance.

Submission Number: 998