Exploring Chain-of-Thought for Multi-modal Metaphor Identification

Anonymous

Abstract: Metaphors are commonly found in advertising and internet memes. However, the free form of internet memes often leads to a lack of high-quality textual data. Metaphor identification demands a deep interpretation of both textual and visual elements, requiring extensive common-sense knowledge, which poses a challenge to language models. To address these challenges, we propose a compact framework that enhances the small model by distilling knowledge from Multi-modal Large Language Models(MLLMS). Specifically, our approach designs a three-step process inspired by Chain-of-Thought (CoT) that extracts and integrates knowledge from larger models into smaller ones. We also developed a modality fusion architecture to transform knowledge from large models into metaphor features, supplemented by auxiliary tasks to improve model performance. Experimental results on the MET-MEME dataset demonstrate that our method not only effectively enhances the metaphor identification capabilities of small models but also outperforms existing models. To our knowledge, this is the first systematic study leveraging MLLMs in metaphor identification tasks.

Paper Type: long

Research Area: Linguistic theories, Cognitive Modeling and Psycholinguistics

Contribution Types: NLP engineering experiment

Languages Studied: english,chinese