Neuro-Symbolic Models for Sentiment Analysis

Jan Kocon, Joanna Baran, Marcin Gruza, Arkadiusz Janz, Michal Kajstura, Przemyslaw Kazienko, Wojciech Korczynski, Piotr Milkowski, Maciej Piasecki, Joanna Szolomicka

Abstract: We propose and test multiple neuro-symbolic methods for sentiment analysis. They combine deep neural networks – transformers and recurrent neural networks – with external knowledge bases. We show that for simple models, adding information from knowledge bases significantly improves the quality of sentiment prediction in most cases. For medium-sized sets, we obtain significant improvements over state-of-the-art transformer-based models using our proposed methods: Tailored KEPLER and Token Extension. We show that the cases with the improvement belong to the hard-to-learn set.