Non-Compositionality in Sentiment: New Data and Analyses

Verna Dankers, Christopher G. Lucas

Submission Type: Regular Short Paper

Submission Track: Linguistic Theories, Cognitive Modeling, and Psycholinguistics

Submission Track 2: Interpretability, Interactivity, and Analysis of Models for NLP

Keywords: sentiment analysis, compositionality, data annotation

TL;DR: We propose a methodology for obtaining non-compositionality ratings of natural language phrases for sentiment analysis. We use that method to create a resource of ratings, present an analysis of that resource, and use it for model evaluation.

Abstract:

When natural language phrases are combined, their meaning is often more than the sum of their parts.
In the context of NLP tasks such as sentiment analysis, where the meaning of a phrase is its sentiment, that still applies.
Many NLP studies on sentiment analysis, however, focus on the fact that sentiment computations are largely compositional.
We, instead, set out to obtain non-compositionality ratings for phrases with respect to their sentiment.
Our contributions are as follows: a) a methodology for obtaining those non-compositionality ratings, b) a resource of ratings for 259 phrases – NonCompSST – along with an analysis of that resource, and c) an evaluation of computational models for sentiment analysis using this new resource.

Submission Number: 5845