Vision Remember: Recovering Visual Information in Efficient LVLM with Vision Feature Resampling

ICLR 2026 Conference Submission18922 Authors

Keywords: Multi-Modality Model, Efficient Large Vision-Language Model

Abstract:

The computational expense of redundant vision tokens in Large Vision-Language Models (LVLMs) has led many existing methods to compress them via a vision projector.
However, this compression may lose visual information that is crucial for tasks relying on fine-grained spatial relationships, such as OCR and Chart \& Table Understanding.
In this paper, we propose to resample original vision features across the LLM decoder layers to recover visual information and attain efficiency.
Following this principle, we introduce Vision Remember, which includes two key modules: (1) Token-Feature Cross-Attention Layer and (2) Token Bidirectional Self-Attention Layer.
In the Token bidirectional attention, we employ self-attention mechanism to maintain the bidirectional interaction between vision tokens and the text-guided token.
In the Token-Feature interaction attention, we introduce local cross-attention to resample the visual feature and utilize the multi-level fusion to enrich the visual representation.
We conduct comprehensive experiments on multiple visual understanding benchmarks and the results with the LLaVA-NeXT baseline show that Vision Remember outperforms TokenPacker by 2.7 and FastV by 5.7 across nearly all the settings.
The experimental results also validate the generalization capability of the proposed method when combined with various efficient vision projectors and LVLMs.

Primary Area: applications to computer vision, audio, language, and other modalities

Submission Number: 18922