Exploring Word Composition Knowledge in Language Usages

Yuchen Han, Tianyuan Liu, Yuqing Sun, Tian Huang, Huiqian Wu, Shengjun Wu

Abstract: The creativity of language is a distinct feature that sets humans apart from machines and animals, where the flexibility of word composition is the fundamental part. The patterns on words that are systematically linked together are acknowledged as the word composition knowledge. We explore this knowledge by combining the syntax information with word semantics and verify it through a series of empirical experiments on multiple datasets. From the linguistic perspective, we found that this knowledge can find the appropriate alternatives for the given phrase and generate high-quality paraphrases that satisfy both the syntax soundness and the semantic consistency with the original text. We also verify it on the questionnaires in psychological testing and find the abnormal patterns on the language usages. Compared to the large pre-trained models, this method is much more training-economic and can generate the paraphrases in an explainable way.