T-MAP: Red-Teaming LLM Agents with Trajectory-aware Evolutionary Search

Hyomin Lee, Sangwoo Park, Yumin Choi, Sohyun An, Hayeon Lee, Seanie Lee, Sung Ju Hwang

Keywords: LLM Agent, Safety, Security, Red-teaming

Abstract: While prior red-teaming efforts have focused on eliciting harmful text outputs of large language models (LLMs), such approaches fail to capture agent-specific vulnerabilities that emerge through multi-step tool execution, particularly in rapidly growing ecosystems such as the model context protocol (MCP). To address this gap, we propose a trajectory-aware evolutionary search method, T-MAP, which leverages execution trajectories to guide the discovery of adversarial prompts. Our approach enables the automatic generation of attacks that not only bypass safety guardrails but also reliably realize harmful objectives through actual tool interactions. Empirical evaluation across diverse MCP environments demonstrates that our method substantially outperforms baselines in attack realization rate, revealing previously underexplored vulnerabilities and aiding in proactively identifying the risks posed by autonomous LLM agents.

Email Sharing: We authorize the sharing of all author emails with Program Chairs.

Data Release: We authorize the release of our submission and author names to the public in the event of acceptance.

Submission Number: 157