Large Language Models for Mental Health: A Multilingual Evaluation

ACL ARR 2025 February Submission2375 Authors

Abstract: Large Language Models (LLMs) have demonstrated impressive capabilities across various NLP tasks, but their performance in multilingual settings is often underexplored. This study evaluates proprietary and open-source LLMs on eight mental health datasets of various languages. We compare their performance in zero-shot, few-shot, and fine-tuned settings against traditional methods. Results show that LLMs achieve competitive or superior F1 scores across several datasets, with fine-tuned models often surpassing state-of-the-art results. However, performance varies across languages, highlighting both the strengths and limitations of LLMs in this critical application. These findings provide actionable insights into the application of LLMs for multilingual mental health text classification.

Paper Type: Short

Research Area: Multilingualism and Cross-Lingual NLP

Research Area Keywords: Multilingualism, Mental Health, Large Language Models

Contribution Types: Model analysis & interpretability, NLP engineering experiment, Data analysis

Languages Studied: Russian, Thai, Bengali, Portuguese, Spanish

Submission Number: 2375