Manual and automatic paragraph-level analysis of climate change framing in academic journal editorials

Nicholas Badullovich, Manfred Stede, Berfin Aktas, Nailia Mirzakhmedova, Patrick Saint-Dizier

Abstract: Framing is an extensively used research method in the context of climate change communication research. Frames are an essential part of communication by which a speaker can put boundaries on complex issues and create a tailored lens through which to communicate while remaining true to reality. Though there is a rich literature looking at frames present in a variety of media and social media content, rather little attention has been given so far to scientific editorials. We extend upon previously published research on climate change related editorials from the journals Nature and Science by extending it with new data, new coding structures, new analyses, and results on automatic classification. Specifically, we show that applying frame analysis not as regularly done on text level but on the level of paragraphs (and moving to the level of individual sentences when necessary) leads to a more nuanced account of the underlying communication strategies. Next, in addition to analysing customary issue frames, we also code the rhetorical dimensions present in the paragraphs, informed by Entman’s four frame functions. In analysing the resulting data, we find a pervasive use of the Governance and Scientific issue frames in both journals, and for the rhetorical dimension a dominance of frames that represent the Describe Problem and the Moral Judgement perspectives. Finally, as an add-on to the qualitative work, we assess the capabilities of automatic text classification methods for our three tasks of determining (i) the degree of paragraph topicality, (ii) the issue frames, and (iii) the rhetorical frames. We compare various supervised und unsupervised methods and find that a RoBERTa model achieves decent performance on the frequent classes, while the rare classes pose problems. Similarly, zero-shot learning with LLMs is not yet able to provide a reliable classification, showcasing the results close to those of an SVM baseline. At the end of the paper, we situate our approach in the quest for even more fine-grained analyses and computational models of framing.

External IDs:dblp:journals/scientometrics/BadullovichSAMS26