POLITICS: Pretraining with Same-story Article Comparison for Ideology Prediction and Stance Detection

Anonymous

Abstract: Ideology is at the core of political science. Yet, there still does not exist general-purpose tools that can characterize and predict ideology across different genres of text. To this end, we study the training of PLMs using novel ideology-driven pretraining objectives that rely on the comparison of articles that are on the same stories but written by media of different ideologies. We further collect a large-scale dataset consisting of more than 3.6M political news articles for experiments. Our model POLITICS and its variants outperform strong baselines on 10 out of the 11 ideology prediction and stance detection tasks. Our analysis further shows that POLITICS is especially good at understanding long or formally written texts, and is also robust in few-shot learning scenarios.