RoMath: A Mathematical Reasoning Benchmark in Romanian

ACL ARR 2025 February Submission4158 Authors

Abstract: Mathematics has long been conveyed through natural language, primarily for human understanding. With the rise of mechanized mathematics and proof assistants, there is a growing need to understand informal mathematical text, yet most existing benchmarks focus solely on English, overlooking other languages. This paper introduces RoMath, a Romanian mathematical reasoning benchmark suite comprising three subsets: Baccalaureate, Competitions and Synthetic, which cover a range of mathematical domains and difficulty levels, aiming to improve non-English language models and promote multilingual AI development. By focusing on Romanian, a low-resource language with unique linguistic features, RoMath addresses the limitations of Anglo-centric models and emphasizes the need for dedicated resources beyond simple automatic translation. We benchmark several open-weight language models, highlighting the importance of creating resources for underrepresented languages. Code and datasets will be made available.

Paper Type: Long

Research Area: Resources and Evaluation

Research Area Keywords: datasets for low resource languages, benchmarking, language resources

Contribution Types: Data resources

Languages Studied: Romanian

Submission Number: 4158