Sentence embedding with contrastive multi-views learning

Antoine Simoulin

TL;DR: We aim to exploit the diversity of linguistic structures to build sentence representations.

Abstract: In this work, we propose a self-supervised method to learn sentence representations with an injection of linguistic knowledge. Multiple linguistic frameworks propose diverse sentence structures from which semantic meaning might be expressed out of compositional words operations. We aim to take advantage of this linguist diversity and learn to represent sentences by contrasting these diverse views. Formally, multiple views of the same sentence are mapped to close representations. On the contrary, views from other sentences are mapped further. By contrasting different linguistic views, we aim at building embeddings which better capture semantic and which are less sensitive to the sentence outward form.

Keywords: contrastive, multi-views, linguistic, embedding