ProReason: Multi-Modal Proactive Reasoning with Decoupled Eyesight and Wisdom

ACL ARR 2025 February Submission3910 Authors

Abstract:

Large vision-language models (LVLMs) have witnessed significant progress on visual understanding tasks.
However, they often prioritize language knowledge over image information on visual reasoning tasks, incurring performance degradation.
To tackle this issue, we first identify the drawbacks of existing solutions (i.e., limited multi-modal reasoning capacities, and insufficient and irrelevant visual descriptions).
We then decompose visual reasoning process into two stages: proactive visual perception (i.e., eyesight) and textual reasoning (i.e., wisdom), and introduce a novel visual reasoning framework named ProReason.
This framework features decoupled vision-reasoning capabilities and multi-run proactive perception.
Briefly, given a multi-modal question, ProReason iterates proactive information collection and reasoning until the answer can be concluded with necessary and sufficient visual descriptions.
Notably, the disassociation of capabilities allows seamless integration of existing large language models (LLMs) to compensate for the reasoning deficits of LVLMs.
Our extensive experiments demonstrate that ProReason outperforms both existing multi-step reasoning frameworks and passive peer methods on various benchmarks for both open-source and closed-source models.
More impressively, with the assistance of LLMs, ProReason achieves a performance improvement of up to 15% on MMMU benchmark.
Our insights into existing solutions and the decoupled perspective for feasible integration of LLMs illuminate future research on visual reasoning techniques, especially LLM-assisted ones.

Paper Type: Long

Research Area: Multimodality and Language Grounding to Vision, Robotics and Beyond

Research Area Keywords: Decoupled vision-reasoning, Proactive reasoning, LLM-assisted multi-modal reasoning, Large vision-language model, Visual reasoning, Multi-step reasoning framework

Contribution Types: NLP engineering experiment

Languages Studied: English

Submission Number: 3910