Deriving Adjectival Scales from Continuous Space Word Representations

Joo-Kyung Kim, Marie-Catherine de Marneffe

Abstract: Continuous space word representations extracted from neural network language models have been used effectively for natural language processing, but until recently it was not clear whether the spatial relationships of such representations were interpretable. Mikolov et al. (2013) show that these representations do capture syntactic and semantic regularities. Here, we push the interpretation of continuous space word representations further by demonstrating that vector offsets can be used to derive adjectival scales (e.g., okay < good < excellent). We evaluate the scales on the indirect answers to yes/no questions corpus (de Marneffe et al., 2010). We obtain 72.8% accuracy, which outperforms previous results ( 60%) on this corpus and highlights the quality of the scales extracted, providing further support that the continuous space word representations are meaningful.