Interpretable word-level context-based sentiment analysis
Primary Area: visualization or interpretation of learned representations
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Keywords: Language Models, Sentiment Analysis, Interpretable Learning
Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.
TL;DR: AMIC is a transparent attention-based model for fine-grained word-level sentiment analysis using only document labels, offering enhanced interpretability while maintaining competitive performance.
Abstract: We propose an attention-based multiple instance classification model (AMIC) to conduct interpretable word-level sentiment analysis (SA) using only document sentiment labels. The word-level SA adds more interpretability compared to other models while maintaining competitive performance at the document level. Furthermore, we decompose our model into interpretable outputs that provide context weighting, indication of word neutrality, and negation. This structure provides insights on how context influences sentiment and the inner workings in the model’s decision-making process. AMIC is built on a straightforward modeling framework (i.e., multiple instance classification model) which incorporates blocks of self-attention and positional encoded self-attention to achieve competitive prediction performance. The architecture is transparent yet effective at conducting interpretable SA. Model performance is reported on two document sentiment classification datasets, with extensive analysis of model interpretation.
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Submission Number: 4259
Loading