Thinking Like a Botanist: Challenging Multimodal Language Models with Intent Driven Chain-of-Inquiry

ACL ARR 2026 January Submission3841 Authors

Keywords: plant disease diagnosis, visual question answering, multi-turn VQA, Chain-of-Inquiry, Reasoning, intent-conditioned reasoning, visual grounding, multimodal datasets, diagnostic reasoning, human-like inquiry, vision-language models

Abstract: Vision evaluations are typically done through multi-step processes. In most contemporary fields, experts analyze images using structured, evidence-based adaptive questioning. In plant pathology, botanists inspect leaf images, identify visual cues, infer diagnostic intent, and probe further with targeted questions that adapt to species, symptoms, and severity. This structured probing is crucial for accurate disease diagnosis and treatment formulation. Yet current vision-language models are evaluated on single-turn question answering. To address this gap, we introduce PlantInquiryVQA, a benchmark for studying multi-step, intent-driven visual reasoning in botanical diagnosis. We formalize a Chain of Inquiry framework modeling diagnostic trajectories as ordered question-answer sequences conditioned on grounded visual cues and explicit epistemic intent. We release a dataset of 24,964 expert-curated plant images and 138,078 question-answer pairs annotated with visual grounding, severity labels, and domain-specific reasoning templates. Evaluations on top-tier Multimodal Large Language Models reveal that while they describe visual symptoms adequately, they struggle with safe clinical reasoning and accurate diagnosis. Importantly, structured question-guided inquiry significantly improves diagnostic correctness, reduces hallucination, and increases reasoning efficiency. PlantInquiryVQA provides a foundation for training diagnostic agents that reason like expert botanists rather than static classifiers.

Paper Type: Long

Research Area: Resources and Evaluation

Research Area Keywords: Multimodal learning, Vision question answering, Benchmarking, Evaluation methodologies, Grounded language understanding

Contribution Types: Model analysis & interpretability, Data resources, Data analysis

Languages Studied: English

Submission Number: 3841