BERT for Sequence-to-Sequence Multi-Label Text Classification

Sep 25, 2019 Withdrawn Submission readers: everyone
  • TL;DR: On using BERT as an encoder for sequential prediction of labels in multi-label text classification task
  • Abstract: We study the BERT language representation model and the sequence generation model with BERT encoder for multi-label text classification task. We experiment with both models and explore their special qualities for this setting. We also introduce and examine experimentally a mixed model, which is an ensemble of multi-label BERT and sequence generating BERT models. Our experiments demonstrated that BERT-based models and the mixed model, in particular, outperform current baselines in several metrics achieving state-of-the-art results on three well-studied multi-label classification datasets with English texts and two private Yandex Taxi datasets with Russian texts.
  • Keywords: Multi-Label Text Classification, Sequence-to-Sequence Learning, BERT, Sequence Generation, Hierarchical Text Classification
  • Original Pdf:  pdf
0 Replies