HUBERT Untangles BERT to Improve Transfer across NLP TasksDownload PDF

25 Sep 2019 (modified: 24 Dec 2019)ICLR 2020 Conference Blind SubmissionReaders: Everyone
  • Original Pdf: pdf
  • TL;DR: We introduce HUBERT which combines the power of Tensor-Product Representations and BERT language model.
  • Abstract: We introduce HUBERT which combines the structured-representational power of Tensor-Product Representations (TPRs) and BERT, a pre-trained bidirectional transformer language model. We validate the effectiveness of our model on the GLUE benchmark and HANS dataset. We also show that there is shared structure between different NLP datasets which HUBERT, but not BERT, is able to learn and leverage. Extensive transfer-learning experiments are conducted to confirm this proposition.
  • Keywords: Tensor Product Representation, BERT, Transfer Learning, Neuro-Symbolic Learning
10 Replies

Loading