Is a Single Embedding Sufficient? Resolving Polysemy of Words from the Perspective of Markov Decision Process

Cheng Zhang, Zhi Chen, Dongmei Yan, Jingxu Cao, Quan Zhang

Abstract: Polysemy is a widespread linguistic phenomenon. A word can carry a variety of semantic information and is recognized as a definite semantic in a specific context. However, in many models that are good at processing text sequences, words are usually treated as a single embedding, which ignores the polysemy characteristics of words and makes it difficult to model the process of semantic cognition. To fill this gap, this paper models a variety of RNNs from the perspective of the Markov decision process (MDP) and classifies them as Single-state RNN (SRNN), pointing out SRNN deficiencies in polysemy and semantic cognitive processes. A Polymorphic Recurrent Neural Network (PRNN) that can effectively simulate the process of human semantic cognition is proposed by improving the policy function. PRNN selects the specific semantics to be expressed according to the actual context in which the word is located. Extensive experimental results show that PRNNs are superior to RNNs in many natural language processing tasks. The analysis of specific cases shows how PRNNs simulate the process of human language cognition.

External IDs:dblp:conf/dasfaa/ZhangCYCZ23