Robust Co-occurrence Quantification for Lexical Distributional Semantics

Dmitrijs Milajevs, Mehrnoosh Sadrzadeh, Matthew Purver

Abstract: Previous optimisations of parameters affecting the word-context association measure used in distributional vector space models have focused either on highdimensional vectors with hundreds of thousands of dimensions, or dense vectors with dimensionality of a few hundreds; but dimensionality of a few thousands is often applied in compositional tasks as it is still computationally feasible and does not require the dimensionality reduction step. We present a systematic study of the interaction of the parameters of the association measure and vector dimensionality, and derive parameter selection heuristics that achieve performance across word similarity and relevance datasets competitive with the results previously reported in the literature achieved by highly dimensional or dense models.