Transferring Textual Preferences to Vision-Language Understanding through Model Merging

ACL ARR 2025 February Submission1280 Authors

Abstract: Large vision-language models (LVLMs) perform outstandingly across various multimodal tasks. However, their ability to evaluate generated content remains limited, and training vision-language reward models (VLRMs) with preference data is computationally expensive. This paper explores a training-free alternative by merging text-based reward models (RMs) with LVLMs to create VLRMs. Our approach shows that integrating these models leads to improved performance over LVLMs' scoring and text-based RMs, offering an efficient method for incorporating textual preferences into LVLMs.

Paper Type: Short

Research Area: Multimodality and Language Grounding to Vision, Robotics and Beyond

Research Area Keywords: transfer learning / domain adaptation, multimodality

Contribution Types: Approaches low compute settings-efficiency

Languages Studied: English

Submission Number: 1280