Multimodal Sentiment Analysis of Social Media Content and Its Impact on Mental Wellbeing: An Investigation of Extreme Sentiments

Rushil Thareja

Abstract: The rapid expansion of social media platforms, such as Twitter, has led to an influx of multimodal content frequently exhibiting extreme emotional sentiments. Continuous exposure to such content can deteriorate a user’s mental health. To mitigate this, we introduce Tweet-SentiNet, a robust multimodal framework employing image and text embeddings for nuanced analysis and prediction of sentiment emotions in social media content. Tweet-SentiNet filters extreme sentiments, thereby reducing exposure to potentially damaging emotional content and aiding platform moderators in identifying victims and instigators of hate, as well as potential cases of depression or suicidal tendencies. The proposed method outperforms existing baselines, demonstrating the significance of multimodal features in determining sentiment. To further assess the mental well-being effects of consuming such data, we conduct a user study with 20 participants and present the findings. This paper sheds light on the potential of utilizing multimodal sentiment analysis for mental health monitoring and intervention on social media platforms.