Towards Automated Situation Awareness: A RAG-Based Framework for Peacebuilding Reports

ACL ARR 2025 July Submission983 Authors

Abstract: Timely and accurate situation awareness underpins effective decision‑making in humanitarian response, conflict monitoring, and peacebuilding. Yet, synthesizing heterogeneous and rapidly evolving information from news, conflict databases, and economic indicators remains labor‑intensive and delays critical interventions. We present a dynamic Retrieval‑Augmented Generation (RAG) system that autonomously produces structured, evidence‑backed situation awareness reports by integrating real‑time data from GDELT, ACLED, ReliefWeb, and World Bank APIs. To rigorously assess report quality without ground‑truth references, we introduce a three‑level reference‑free evaluation framework combining automated NLP metrics, expert review by United Nations crisis analysts, and scalable LLM‑as‑a‑Judge assessment. In a multi‑country study across 15 conflict‑prone regions, our system generated coherent, relevant, and actionable reports, reducing analyst preparation time by nearly 50\%. These findings demonstrate the feasibility of deploying RAG‑based systems for peacebuilding and humanitarian operations and provide a reproducible framework for generating and evaluating AI‑assisted situational intelligence at scale.

Paper Type: Long

Research Area: Computational Social Science and Cultural Analytics

Research Area Keywords: retrieval-augmented generation, domain adaptation, data-to-text generation, NLP for social good, humanitarian applications, quantitative analyses of news and social media, NLP tools for social analysis

Contribution Types: Model analysis & interpretability, NLP engineering experiment

Languages Studied: English

Reassignment Request Area Chair: This is not a resubmission

Reassignment Request Reviewers: This is not a resubmission

A1 Limitations Section: This paper has a limitations section.

A2 Potential Risks: Yes

A2 Elaboration: Ethic Section

B Use Or Create Scientific Artifacts: Yes

B1 Cite Creators Of Artifacts: Yes

B1 Elaboration: All used model are cited throughout the paper

B2 Discuss The License For Artifacts: N/A

B3 Artifact Use Consistent With Intended Use: N/A

B4 Data Contains Personally Identifying Info Or Offensive Content: N/A

B5 Documentation Of Artifacts: N/A

B6 Statistics For Data: Yes

B6 Elaboration: Section Method

C Computational Experiments: Yes

C1 Model Size And Budget: Yes

C1 Elaboration: Section Method

C2 Experimental Setup And Hyperparameters: N/A

C3 Descriptive Statistics: Yes

C3 Elaboration: Section Results

C4 Parameters For Packages: Yes

C4 Elaboration: Section Method

D Human Subjects Including Annotators: Yes

D1 Instructions Given To Participants: Yes

D1 Elaboration: Section Evaluation

D2 Recruitment And Payment: Yes

D2 Elaboration: Section Evaluation

D3 Data Consent: N/A

D4 Ethics Review Board Approval: N/A

D5 Characteristics Of Annotators: Yes

D5 Elaboration: Section Evaluation

E Ai Assistants In Research Or Writing: Yes

E1 Information About Use Of Ai Assistants: Yes

E1 Elaboration: Ethics section

Author Submission Checklist: yes

Submission Number: 983