Let Androids Dream of Electric Sheep: A Human-Inspired Image Implication Understanding and Reasoning Framework

Chenhao Zhang, Yazhe Niu

Keywords: Image Implication, Image Metaphor, Visual Question Answering, Vision and Language, Vision Language Model

Abstract:

Metaphorical comprehension in images remains a critical challenge for AI systems, as existing models struggle to grasp the nuanced cultural, emotional, and contextual implications embedded in visual content.
While multimodal large language models (MLLMs) excel in basic Visual Question Answer (VQA) tasks, they exhibit a fundamental limitation on image implication tasks: contextual gaps that obscure the relationships between different visual elements and their abstract meanings.
Inspired by the human cognitive process, we propose ***Let Androids Dream (LAD)***, a novel framework for image implication understanding and reasoning.
LAD addresses contextual missing through the three-stage framework: (1) **Perception**: converting visual information into rich and multi-level textual representations, (2) **Search**: iteratively searching and integrating cross-domain knowledge to resolve ambiguities, and (3) **Reasoning**: generating contextual-alignment image implication via explicit reasoning.
Our framework with the lightweight GPT-4o-mini model achieves SOTA performance compared to 15+ MLLMs on English image implication benchmark and a huge improvement on Chinese benchmark, performing comparable with the GPT-4o model on Multiple-Choice Question (MCQ) and outperforms 36.7\% on Open-Style Question (OSQ). Additionally, our work provides new insights into how AI can more effectively interpret image implications, advancing the field of vision-language reasoning and human-AI interaction. Our project is publicly available at https://anonymous.4open.science/r/Let-Androids-Dream-of-Electric-Sheep.

Primary Area: applications to computer vision, audio, language, and other modalities

Submission Number: 12234