A Comparative Analysis of Stance Detection Approaches and Datasets

Parush Gera

Abstract:

Various approaches have been proposed for automated stance detection, including those that
use machine and deep learning models and natural language processing techniques. However,
their cross-dataset performance, the impact of
sample size on performance, and experimental
aspects such as runtime have yet to be compared, limiting what is known about the generalizability of prominent approaches. This paper presents a replication study of stance detection approaches on current benchmark datasets.
Specifically, we compare six existing machine
and deep learning stance detection models on
three publicly available datasets. We investigate performance as a function of the number
of samples, length of samples (word count),
representation across targets, type of text data,
and the stance detection models themselves.
We identify the current limitations of these approaches and categorize their utility for stance
detection under varying circumstances (e.g.,
size of text samples), which provides valuable
insight for future research in stance detection.