Unified Representation for Non-compositional and Compositional Expressions
Submission Type: Regular Long Paper
Submission Track: Language Modeling and Analysis of Language Models
Submission Track 2: Semantics: Lexical
Keywords: Potentially Idiomatic Expression; Non-compositionality; Phrase Embedding; Idiomatic Expression Processing
TL;DR: We propose a new method to produce semantically meaningful and contextually appropriate embeddings for potentially idiomatic expressions that benefits the performances of the downstream idiomatic processing tasks.
Abstract: Accurate processing of non-compositional language relies on generating good representations for such expressions. In this work, we study the representation of language non-compositionality by proposing a language model, PIER+, that builds on BART and can create semantically meaningful and contextually appropriate representations for English potentially idiomatic expressions (PIEs). PIEs are characterized by their non-compositionality and contextual ambiguity in their literal and idiomatic interpretations. Via intrinsic evaluation on embedding quality and extrinsic evaluation on PIE processing and NLU tasks, we show that representations generated by PIER+ result in 33\% higher homogeneity score for embedding clustering than BART, whereas 3.12\% and 3.29\% gains in accuracy and sequence accuracy for PIE sense classification and span detection compared to the state-of-the-art IE representation model, GIEA. These gains are achieved without sacrificing PIER+'s performance on NLU tasks (+/- 1\% accuracy) compared to BART.
Submission Number: 2147
Loading