ProgressLM: Towards Progress Reasoning in Vision-Language Models

ACL ARR 2026 January Submission1088 Authors

Keywords: Vision-Language Models, Progress Reasoning

Abstract:

Estimating task progress requires long-horizon and dynamic reasoning, going beyond static visual perception.
Although Vision-Language Models (VLMs) excel at describing what is visible in a single observation, it remains unclear whether they can infer how far a task has progressed from partial information.
To study this question, we introduce Progress-Bench, a benchmark with over 3K instances for evaluating progress reasoning from a single observation.
We further examine a human-inspired two-stage paradigm that combines episodic retrieval with mental simulation.
We instantiate this paradigm through both training-free prompting and a training-based approach using the automatically curated ProgressLM-45K dataset.
Experiments on 14 VLMs show that most models struggle with reliable progress estimation, and that training-free reasoning provides only limited and model-dependent benefits.
In contrast, the training-based ProgressLM-3B achieves consistent improvements in accuracy, robustness to viewpoint variation, and handling of unanswerable cases, despite its small scale.
Additional analyses reveal common failure patterns in existing VLMs and clarify when and why progress reasoning succeeds or fails.

Paper Type: Long

Research Area: Multimodality and Language Grounding to Vision, Robotics and Beyond

Research Area Keywords: vision question answering, cross-modal application, multimodality

Contribution Types: Model analysis & interpretability, NLP engineering experiment, Data resources, Data analysis

Languages Studied: English

Submission Number: 1088