SDG target detection in environmental reports using Retrieval-augmented Generation with LLMs

Dario Garigliotti

Keywords: Retrieval-augmented Generation, Large Language Models, SDG targets, Environmental Impact Assessment

TL;DR: A pair of datasets and an evaluation of RAG-based methods with LLMs for the tasks of SDG target evidence identification and SDG target detection in environmental reports.

Abstract:

With the consolidation of Large Language Models (LLM) as a dominant component in approaches for multiple linguistic tasks, the interest in these technologies has greatly increased within a variety of areas and domains.
A particular scenario of information needs where to exploit these approaches is climate-aware NLP.
Paradigmatically, the vast manual labour of inspecting long, heterogeneous documents to find environment-relevant expressions and claims suits well within a recently established Retrieval-augmented Generation (RAG) framework.
In this paper, we tackle two dual problems within environment analysis dealing with the common goal of detecting a Sustainable Developmental Goal (SDG) target being addressed in a textual passage of an environmental assessment report.
We develop relevant test collections, and propose and evaluate a series of methods within the general RAG pipeline, in order to assess the current capabilities of LLMs for the tasks of SDG target evidence identification and SDG target detection.

Archival Submission: arxival

Submission Number: 35