SudoBench: A Contextual Authorization Benchmark for LLM Agents

Vincent Siu, Tianneng Shi, Shangding Gu, Zhun Wang, Dawn Song, Chenguang Wang

Keywords: IFC, security, agents, prompt injection

Abstract: Secure LLM agents should take actions for authorized users in authorized environments. However, current agents' (e.g., OpenClaw) runtimes expose neither user nor environment authorization and existing agent security benchmarks similarly evaluate models without relevant authorization context. We propose SudoBench, a benchmark of 135 paired scenarios in a synthetic consulting universe with a static access control list, spanning user authorization (confused deputy), environment authorization (source-label indirect prompt injection), and both (scope-check indirect prompt injection). For each pair of scenarios, the authorization context is flipped such that an identical prompt should be appropriately complied with or refused. We score each pair as a joint pass requiring the model to produce the correct outcome on both paired scenarios. Across ten frontier closed- and open-weight LLMs, joint pass rate without authorization context sits below 16% on every model. Adding user and environment authorization raises joint pass to as high as 67--73%. Even with this authorization context, frontier models remain far from reliably handling contextual authorization, framing authorization context as an important research agenda for agent security.

Track: Regular Paper (9 pages)

Email Sharing: We authorize the sharing of all author emails with Program Chairs.

Data Release: We authorize the release of our submission and author names to the public in the event of acceptance.

Submission Number: 146