Uncovering Latent Arguments in Social Media Messaging by Employing LLMs-in-the-Loop Strategy

Tunazzina Islam, Dan Goldwasser

Abstract:

The widespread use of social media has led to
a surge in popularity for automated methods of
analyzing public opinion. Supervised methods
are adept at text categorization, yet the dynamic
nature of social media discussions poses a continual challenge for these techniques due to the
constant shifting of the focus. On the other
hand, traditional unsupervised methods for extracting themes from public discourse, such as
topic modeling, often reveal overarching patterns that might not capture specific nuances.
Consequently, a significant portion of research
into social media discourse still depends on
labor-intensive manual coding techniques and
a human-in-the-loop approach, which are both
time-consuming and costly. In this work, we
study the problem of discovering arguments associated with a specific theme. We propose a
generic LLMs-in-the-Loop strategy that leverages the advanced capabilities of large language models (LLMs) to extract latent arguments from social media messaging. To demonstrate our approach, we apply our framework to
contentious topics. We use two publicly available datasets: (1) the climate campaigns dataset
of 14k Facebook ads with 25 themes and (2)
the COVID-19 vaccine campaigns dataset of 9k
Facebook ads with 14 themes. Additionally, we
design a downstream task as stance prediction
by leveraging talking points in climate debates.
Furthermore, we analyze demographic targeting and the adaptation of messaging based on
real-world events.