Text Labels Classification Model based on BERT Algorithm

Sai Wu, Zhiyin Huang, Hao Feng

Abstract: Text labels are extracted from the content of texts and is a issue of natural language process, which contains multiple labels. The text labels classification aims to divide the multiple labels into only one correct category. Traditionally, researchers utilize machine learning algorithms to train an one classifier to solve the multiple labels, which may cause wrong classifier due to the machine learning can not process the relationship between texts. In this paper, we utilize pre-training of deep bidirectional transformers (BERT) to dispose text understanding issues. Initially, the pre-train process is established to capture the text representations and the initial classification is executed. Subsequently, the fine-tunning function is to optimize the parameters of BERT model including learning rate, training epochs and batch sizes. From our extensive experimental results, we can significantly conclude that our proposed method can achieve the text labels classification with reasonable computation costs and training process.

External IDs:dblp:conf/icbase/WuHF23