Challenges in Using LLM Agents to Validate Agent Governance

Ayana Hussain, Ulrich Aïvodji, Héber Hwang Arcolezi

Keywords: Agents, Governance, Simulations

TL;DR: This paper explores how LLM simulation challenges apply to agent governance and proposes solutions and an example simulation system design.

Abstract: The increasing deployment of Large Language Models (LLMs) as autonomous agents has intensified the need for credible and trustworthy methods to evaluate governance interventions. Motivated by recent research, this work considers the use of LLM and agent-based simulations to evaluate AI agent governance mechanisms before real-world deployment. While conceptually appealing, this approach introduces various challenges. We examine three such problems: (1) obtaining ground truth for validation, (2) determining whether observed behaviors represent actual agent operations or simulation artifacts, and (3) obtaining consent for data use, and addressing ethical concerns about computational surrogates replacing real users. We also outline considerations based on documented limitations, aiming to catalyze workshop discussion on trustworthy and reliable evaluation methods for agent governance.

Email Sharing: We authorize the sharing of all author emails with Program Chairs.

Data Release: We authorize the release of our submission and author names to the public in the event of acceptance.

Submission Number: 8