Jointly Learning Sentence Embeddings and Syntax with Unsupervised Tree-LSTMs

Jean Maillard, Stephen Clark, Dani Yogatama

Feb 15, 2018 (modified: Oct 27, 2017) ICLR 2018 Conference Blind Submission readers: everyone Show Bibtex
  • Abstract: We introduce a neural network that represents sentences by composing their words according to induced binary parse trees. We use Tree-LSTM as our composition function, applied along a tree structure found by a fully differentiable natural language chart parser. Our model simultaneously optimises both the composition function and the parser, thus eliminating the need for externally-provided parse trees which are normally required for Tree-LSTM. It can therefore be seen as a tree-based RNN that is unsupervised with respect to the parse trees. As it is fully differentiable, our model is easily trained with an off-the-shelf gradient descent method and backpropagation. We demonstrate that it achieves better performance compared to various supervised Tree-LSTM architectures on a textual entailment task and a reverse dictionary task. Finally, we show how performance can be improved with an attention mechanism which fully exploits the parse chart, by attending over all possible subspans of the sentence.
  • TL;DR: Represent sentences by composing them with Tree-LSTMs according to automatically induced parse trees.
  • Keywords: hierarchical, tree-lstm, treelstm, syntax, composition
0 Replies