Plutus: Benchmarking Large Language Models in Low-Resource Greek Finance

ACL ARR 2025 May Submission5131 Authors

Abstract: Despite Greece’s pivotal role in the global economy, large language models (LLMs) remain underexplored for Greek financial context due to the linguistic complexity of Greek and the scarcity of domain-specific datasets. While multilingual financial NLP has revealed large performance gaps across languages, no benchmarks or LLMs have been tailored for Greek financial tasks until now. To bridge this gap, we introduce Plutus-ben, the first Greek Financial Evaluation Benchmark, and Plutus-8B, the first financial LLM fine-tuned on Greek-specific financial data. Plutus-ben addresses five core tasks: numeric/textual named entity recognition, question answering, abstractive summarization, and topic classification. To support these tasks, we release three new expert-annotated Greek financial datasets and incorporate two existing resources. Our comprehensive evaluation of 22 LLMs reveals persistent challenges in Greek financial NLP, driven by linguistic complexity, domain terminology, and financial reasoning gaps. Experiment results underscore the limitations of cross-lingual transfer and the need for Greek-specific financial modeling. We publicly release Plutus-ben, Plutus-8B, and all associated datasets to promote reproducible research and advance multilingual financial NLP.

Paper Type: Long

Research Area: Multilingualism and Cross-Lingual NLP

Research Area Keywords: large language models,financial domain,cross-lingual NLP

Contribution Types: Data resources, Data analysis

Languages Studied: English,Greek

Submission Number: 5131