Skip-Gram - Zipf + Uniform = Vector Additivity

Alex Gittens, Dimitris Achlioptas, Michael W. Mahoney

Abstract: In recent years word-embedding models have gained great popularity due to their remarkable performance on several tasks, including word analogy questions and caption generation. An unexpected “side-effect” of such models is that their vectors often exhibit compositionality, i.e., adding two word-vectors results in a vector that is only a small angle away from the vector of a word representing the semantic composite of the original words, e.g., “man” + “royal”