Are (LLM) Sentence Transformers really useful for unsupervised hospitality opinion mining?

ACL ARR 2025 February Submission4404 Authors

Abstract:

Despite growing interest in opinion mining for
the hospitality industry, the lack of benchmarks
aligned with real-world use cases limits the
development of robust classifiers. Additionally, recent advancements in dense retrieval
methods using Sentence Transformers, which
enable zero-shot text classification, have not
been thoroughly explored. This study evaluates
embedding models for classifying hospitality reviews
using publicly available human-annotated datasets to assess their limitations
and applicability for opinion mining. Our findings indicate that
dense retrieval models based
on large language models either underperform
or show only marginal improvements over a
simple continuous bag of words model trained
on in-domain data. While fine-tuning pre-trained sentence transformers
perform strongly in extracting both sentiment
and topic information, the lack of sufficient training data limits
the development of effective solutions. Finally,
we offer recommendations, based on key surveys
in the literature, to bridge the gap between
domain-specific needs and recent NLP advancements,
thereby enhancing opinion mining in the
hospitality sector.

Paper Type: Short

Research Area: Sentiment Analysis, Stylistic Analysis, and Argument Mining

Research Area Keywords: Information Retrieval, Text Mining, Sentiment Analysis

Contribution Types: Model analysis & interpretability, Approaches low compute settings-efficiency, Position papers

Languages Studied: English

Submission Number: 4404