AMO-Bench: Large Language Models Still Struggle in High School Math Competitions

ACL ARR 2026 January Submission282 Authors

Keywords: Large Language Models, Reasoning Models, Mathematical Reasoning, Math Evaluation

Abstract:

We present **AMO-Bench**, an **A**dvanced **M**athematical reasoning benchmark with **O**lympiad level or even higher difficulty, comprising 50 human-crafted problems.
Existing benchmarks have widely leveraged high school math competitions for evaluating mathematical reasoning capabilities of large language models (LLMs).
However, many existing math competitions are becoming less effective for assessing top-tier LLMs due to performance saturation (e.g., AIME24/25).
To address this, AMO-Bench introduces more rigorous challenges by ensuring all 50 problems are (1) cross-validated by experts to meet at least the International Mathematical Olympiad (IMO) difficulty standards, and (2) entirely original problems to prevent potential performance leakages from data memorization.
Experimental results across 36 LLMs on AMO-Bench highlights three key findings:
(1) high-level mathematical reasoning remains challenging for current LLMs, with even the best-performing model achieving only 63.1% accuracy and most LLMs scoring below 50%;
(2) scaling test-time compute remains a highly effective strategy for substantially improving reasoning performances,
and (3) open-source models are progressively narrowing the performance gap with proprietary models.
Additionally, we conduct further analysis about reasoning efficiency, volatility, and cross-lingual robustness, providing deeper insights behind the reasoning performances.

Paper Type: Long

Research Area: Mathematical, Symbolic, Neurosymbolic, and Logical Reasoning

Research Area Keywords: Mathematical reasoning

Contribution Types: Model analysis & interpretability, NLP engineering experiment, Publicly available software and/or pre-trained models, Data resources, Data analysis

Languages Studied: English

Submission Number: 282