Go to ACL ARR 2025 February homepageA Couch Potato is not a Potato on a Couch: Prompting Strategies, Image Generation, and Compositionality Prediction for Noun Compounds
Abstract: Predicting the compositionality of English noun-noun compounds, such as climate change and couch potato, has traditionally relied on text-based methods. We explore a novel image-based approach, believing that images convey rich information beyond what text alone can capture and that visual context may provide valuable insights. We generate images for compounds and their constituents using variants of text prompts, then encode these images with Vision Transformers, and assess the depicted meaning relatedness through cosine similarity. Evaluated against human compositionality ratings, the image-based approach performs en par with text-based methods for concrete compounds, while challenges in image acquisition and the misalignment between visual and semantic similarity negatively affect the results for abstract compounds.
Paper Type: Short
Research Area: Semantics: Lexical and Sentence-Level
Research Area Keywords: compositionality, multi-word expressions, word embeddings
Contribution Types: Data resources, Data analysis
Languages Studied: English
Submission Number: 1876
Loading