FinMR: A Comprehensive Benchmark for Multimodal Financial Reasoning with Insights from Error Feedback Learning

ACL ARR 2025 May Submission579 Authors

Abstract:

We introduce **FinMR**, a novel multimodal benchmark designed to evaluate the reasoning capabilities of multimodal LLMs in financial problem-solving. **FinMR** features 3,200 college-level question-answer pairs, 1,049 focused on financial math and 2,151 on financial expertise, integrating textual and visual content, such as stock price trends. To enhance financial reasoning, we employ Error Feedback Learning (**EFL**), which incorporates negative examples and feedback for iterative improvement. Through evaluations of open-source and closed-source models, we demonstrate that MLLMs outperform LLMs and that **FinMR** is more effective than CoT prompting. Our error analysis highlights key challenges in image recognition, question understanding, and formula application, providing insights for future research. **FinMR** establishes a robust foundation for advancing financial reasoning capabilities and developing more effective multimodal reasoning techniques.
Our code and data can be found at [GitHub](https://anonymous.4open.science/r/FinMR-1FDD/README.md) ([https://anonymous/FinMR/Code\&Data](https://anonymous/FinMR/Code&Data)). The leaderboard can be found at [Leaderboard](https://anonymous.4open.science/w/FinMR-homepage-35EF/) ([https://anonymous/FinMR/Leaderboard](https://anonymous/FinMR/Leaderboard)).

Paper Type: Long

Research Area: Multimodality and Language Grounding to Vision, Robotics and Beyond

Research Area Keywords: Financial reasoning, Error Feedback Learning, Multimodal Reasoning

Contribution Types: Data resources, Data analysis

Languages Studied: English

Keywords: Financial reasoning, Error Feedback Learning, Multimodal Reasoning

Submission Number: 579