Probing Semantic Alignment, Lexical Invariance, and Syntactic Influence in LLM Metaphor Processing

ACL ARR 2026 January Submission9689 Authors

Keywords: metaphor, natural language explanations, sentence embedding

Abstract: Large language models (LLMs) achieve strong performance on metaphor detection and interpretation tasks, yet it remains unclear what such behavioral success actually reveals about metaphor processing. We present a diagnostic analysis that examines the limits of behavioral evidence by probing three complementary dimensions: semantic attribute alignment, lexical invariance, and syntactic sensitivity. Using geometric probing, we assess whether model-generated interpretations align with reference semantic attributes; through context-varying substitution, we analyze the stability of lexical associations between metaphorical and literal expressions; and via controlled syntactic perturbations, we examine sensitivity in metaphor detection. Our analysis reveals that, LLM-generated interpretations can exhibit semantic drift relative to reference attributes; stable lexical anchors persist across contextual conditions, potentially supporting conventional metaphors while biasing novel metaphors requiring contextual integration; and detection performance is sensitive to syntactic irregularities. These findings suggest that strong behavioral performance may reflect heterogeneous underlying signals, highlighting the need for caution when interpreting metaphor benchmarks as evidence of robust, integrated semantic understanding.

Paper Type: Long

Research Area: Interpretability and Analysis of Models for NLP

Research Area Keywords: metaphor, natural language explanations, sentence embedding

Contribution Types: Model analysis & interpretability, Data analysis

Languages Studied: English

Submission Number: 9689