Interpreting Attention Heads for Image-to-Text Information Flow in Large Vision–Language Models

Jinyeong Kim, Seil Kang, Jiwoo Park, Junhyeok Kim, Seong Jae Hwang

Keywords: Causal interventions

Other Keywords: Large Vision-Language Models, Visual Object Identification, Attention Heads

TL;DR: The image-to-text transfer in LVLMs depends on specialized attention head groups that are determined by the semantic content of the image.

Abstract:

Large Vision-Language Models (LVLMs) answer visual questions by transferring information from images to text through a series of attention heads. While this image-to-text information flow is central to visual question answering, its underlying mechanism remains difficult to interpret due to the simultaneous operation of numerous attention heads.
To address this challenge, we propose *head attribution*, a technique inspired by component attribution methods, to identify consistent patterns among attention heads that play a key role in information transfer. Using head attribution, we investigate how LVLMs rely on specific attention heads to identify and answer questions about the main object in an image.
Our analysis reveals that a distinct subset of attention heads facilitates the image-to-text information flow.
Remarkably, we find that the selection of these heads is governed by the semantic content of the input image rather than its visual appearance.
We further examine the flow of information at the token level and discover that (1) text information first propagates to role-related tokens and the final token before receiving image information, and (2) image information is embedded in both object-related and background tokens.
Our work provides evidence that image-to-text information flow follows a structured process, and that analysis at the attention-head level offers a promising direction toward understanding the mechanisms of LVLMs.

Submission Number: 107