SentimentPulse: Real-Time Consumer Sentiment Analysis in Custom Language Models

Anonymous

TL;DR: We propose a relative small custom language model for consumer sentiment analysis that can outperform GPT-3.5 and GPT-4

Abstract: Large Language Models are trained on an extremely large corpus of text data to allow better generalization but this blessing can also become a curse and significantly limit their performance in a subset of tasks. In this work, we argue that LLMs are notably behind well-tailored and specifically designed models where the temporal aspect is important in making decisions and the answer depends on the timespan of available training data. We prove our point by comparing two major architectures: first, SentimentPulse, a real-time consumer sentiment analysis approach that leverages custom language models and continual learning techniques, and second, GPT-3.5-Turbo and GPT-4 (GPTs) which are both tested on the same data. Unlike foundation models, which lack temporal context, our custom language model is pre-trained on time-stamped data, making it uniquely suited for real-time application. Additionally, we employ continual learning techniques to pre-train the proposed model, and then use classification and contextual multi-arm bandits to fine-tune, enhancing its adaptability and performance over time. We present a comparative analysis of the predictions accuracy of both proposed architecture and GPTs models. To the best of our knowledge, this is the first application of custom language models for real-time consumer sentiment analysis beyond the scope of conventional surveys.

Paper Type: long

Research Area: NLP Applications

Contribution Types: NLP engineering experiment

Languages Studied: English