Learning Multilingual Idiomatic Representations by an Adaptive Contrastive Triplet Loss

Anonymous

Abstract:

Accurately modeling idiomatic or non-compositional language has been a longstanding challenge in Natural Language Processing (NLP).
This is partly because these expressions do not derive their meanings solely from their constituent words, but also due to the scarcity of relevant data resources, and their impact on the performance of downstream tasks such as machine translation and simplification. In this paper we propose an approach to model idiomaticity effectively using a triplet loss that incorporates the asymmetric contribution of components words to an idiomatic meaning for training language models by using adaptive contrastive learning and resampling miners to build an idiomatic-aware learning objective. Our proposed method is evaluated on a SemEval challenge and outperforms previous alternatives significantly in many metrics.

Paper Type: long

Research Area: Linguistic theories, Cognitive Modeling and Psycholinguistics

Contribution Types: NLP engineering experiment, Approaches to low-resource settings, Approaches low compute settings-efficiency

Languages Studied: English, Portuguese

Preprint Status: There is no non-anonymous preprint and we do not intend to release one.

A1: yes

A1 Elaboration For Yes Or No: 9

A2: no

A3: yes

A3 Elaboration For Yes Or No: 1

B: no

B1: n/a

B2: n/a

B3: n/a

B4: n/a

B5: n/a

B6: no

B6 Elaboration For Yes Or No: The dataset is publicly well-known, and we cite its resource paper.

C: yes

C1: yes

C1 Elaboration For Yes Or No: 4

C2: yes

C2 Elaboration For Yes Or No: 4

C3: yes

C3 Elaboration For Yes Or No: 4

C4: yes

D: no

D1: n/a

D2: n/a

D3: n/a

D4: n/a

D5: n/a

E: no

E1: n/a