Farsi Review Rating Prediction Using Prompt Engineering and RAG

Vanooshe nazari, Alireza Moradi, Sauleh Eetemadi, Mohammad Fathian

Keywords: Rating Prediction, Large Language Models, Retrieval Augmented Generation, Zero-shot Learning, Few-shot Learning

TL;DR: This study uses large language models, namely GPT-3.5 Turbo and Dorna, for review rating prediction with a Farsi dataset. The methods, including zero-shot, few-shot, and Retrieval-Augmented Generation, outperformed the baselines.

Abstract: Review rating prediction is a crucial task that benefits both businesses and customers by enhancing decision-making and improving service quality. In this study, we propose several prompt-based methods for predicting rating of reviews using large language models, specifically GPT-3.5 Turbo and Dorna. We utilize BaSalam dataset, sourced from an Iranian online marketplace. Our approach includes zero-shot and few-shot prompting, as well as Retrieval-Augmented Generation. We evaluate the effectiveness of our methods by comparing them to baseline models, demonstrating superior performance in terms of mean absolute error (MAE) and mean squared error (MSE).

Submission Number: 55