From Word Embeddings To Document Distances

Matt J. Kusner, Yu Sun, Nicholas I. Kolkin, Kilian Q. Weinberger

Abstract: We present the Word Mover’s Distance (WMD),a novel distance function between text docu-ments. Our work is based on recent results inword embeddings that learn semantically mean-ingful representations for words from local co-occurrences in sentences. The WMD distancemeasures the dissimilarity between two text doc-uments as the minimum amount of distance thatthe embedded words of one document need to“travel” to reach the embedded words of anotherdocument. We show that this distance metric canbe cast as an instance of the Earth Mover’s Dis-tance, a well studied transportation problem forwhich several highly efficient solvers have beendeveloped. Our metric has no hyperparametersand is straight-forward to implement. Further, wedemonstrate on eight real world document classi-fication data sets, in comparison with seven state-of-the-art baselines, that the WMD metric leadsto unprecedented lowk-nearest neighbor docu-ment classification error rates