GRATR: Zero-Shot Evidence Graph Retrieval-Augmented Trustworthiness Reasoning

ACL ARR 2025 May Submission3119 Authors

Abstract: Trustworthiness reasoning aims to enable agents in multiplayer games with incomplete information to identify potential allies and adversaries, thereby enhancing decision-making. In this paper, we introduce the graph retrieval-augmented trustworthiness reasoning (GRATR) framework, which retrieves observable evidence from the game environment to inform decision-making by large language models (LLMs) without requiring additional training, making it a zero-shot approach. Within the GRATR framework, agents first observe the actions of other players and evaluate the resulting shifts in inter-player trust, constructing a corresponding trustworthiness graph. During decision-making, the agent performs multi-hop retrieval to evaluate trustworthiness toward a specific target, where evidence chains are retrieved from multiple trusted sources to form a comprehensive assessment. Experiments in the multiplayer game Werewolf demonstrate that GRATR outperforms the alternatives, improving reasoning accuracy by 50.5% and reducing hallucination by 30.6% compared to the baseline method. Additionally, when tested on a dataset of Twitter tweets during the U.S. election period, GRATR surpasses the baseline method by 10.4% in accuracy, highlighting its potential in real-world applications such as intent analysis.

Paper Type: Long

Research Area: NLP Applications

Research Area Keywords: Large Language Model, LLM Applications, Agent, Retrieval Augmented Generation

Languages Studied: English

Submission Number: 3119