Small Language Models for Emotion Recognition in Polish Stock Market Investor Opinions

Bartlomiej Koptyra, Marcin Oleksy, Ewa Dzieciol, Jan Kocon

Abstract: In this paper, we explore the application of small language models for emotion recognition in Polish stock market investor opinions. Emotion recognition has been shown to enhance stock price prediction models by providing meaningful text features. We utilize publicly available pre-trained transformer models and fine-tune them for emotion classification in Polish business articles related to WIG20, the Polish equivalent of S&P 500. Given the scarcity of domain-specific pre-trained models for Polish, we experiment with different transformer architectures, comparing their performance in recognizing emotions such as anger, anticipation, joy, sadness, and trust. Our findings indicate that the choice of a pre-trained model significantly affects performance, with the Polish RoBERTa model yielding the best results for both sentence and document-level emotion classification. We also discuss the challenges of class imbalance and the potential for improving results through additional pre-training on domain-specific data. This work contributes to developing emotion classification models for financial text in Polish and improving stock market prediction tasks.