Open Peer Review. Open Publishing. Open Access. Open Discussion. Open Directory. Open Recommendations. Open API. Open Source.
Infinite Dimensional Word Embeddings
Eric Nalisnick, Sachin Ravi
Feb 15, 2017 (modified: Feb 15, 2017)ICLR 2017 workshop submissionreaders: everyone
Abstract:We describe a method for learning word embeddings with data-dependent dimensionality. Our Infinite Skip-Gram (iSG) and Infinite Continuous Bag-of-Words (iCBOW) are nonparametric analogs of Mikolov et al.'s (2013) well-known 'word2vec' models. Vectors are made infinite dimensional by employing techniques used by Cote & Larochelle (2016) to define a RBM with an infinite number of hidden units. We show qualitatively and quantitatively that the iSG and iCBOW are competitive with their fixed-dimension counterparts while having the ability to infer the appropriate capacity of each word representation.
TL;DR:Word embeddings with stochastic dimensionality.
Keywords:Natural language processing, Unsupervised Learning
Enter your feedback below and we'll get back to you as soon as possible.