A Character-level Short Text Classification Model Based On Spiking Neural Networks

Chengzhi Jiang, Linjing Li, Daniel Dajun Zeng, Xiaoxuan Wang

Abstract: Spiking Neural Networks (SNNs), also referred to as the third generation of artificial neural networks, are highly prized for their biological realism, robustness, and low power requirements. SNNs are crucial in fields such as object detection, image recognition, etc. The classification of short text plays an significant role in the development of chatbots and intent detection. It is also an important task that is widely used in many downstream tasks. However, studies applying SNNs to short text classification are limited. This paper provides a new model that uses SNNs to classify short texts. SNNs are difficult to train directly when using deep models and cannot employ large-scale language models to learn good embeddings. To resolve the challenge, we apply the character-level encoding method and convert analog neural networks into SNNs. To begin with, we represent character-level text using a temporal-and-rate joint horizontal encoding method. Then we develop a tailored deep Convolutional Neural Network (CNN) model for classifying texts. At the inference stage, we convert the tailored CNN model into an SNN model. To test the effectiveness of the proposed method, we conduct text encoding experiments on the NAMES dataset and short text classification experiments on both the 20-newsgroups dataset and the emoji-mult dataset. Experiments demonstrate that the proposed method can obtain classification accuracies that are better than or comparable to other methods.

External IDs:dblp:conf/ijcnn/JiangLZW23