Towards Policy-Compliant Agents: Learning Efficient Guardrails for Policy Violation Detection

Xiaofei Wen, Wenjie Jacky Mo, Yanan Xie, Peng Qi, Muhao Chen

Keywords: Web Agents, Guardrails, Policy Compliance

TL;DR: We introduce TrajGuardBench, a 60k-scale benchmark for detecting policy–trajectory violations in web agents, and train PolicyGuard-4B, a lightweight guardrail that outperforms larger baselines while generalizing across domains.

Abstract:

Autonomous web agents need to operate under externally imposed or human-specified policies while generating long-horizon trajectories.
However, little work has examined whether these trajectories comply with such policies, or whether policy violations persist across different contexts such as domains (e.g., shopping or coding websites) and subdomains (e.g., product search and order management in shopping).
To address this gap, we introduce PolicyGuardBench, a benchmark of about 60k examples for detecting policy violations in agent trajectories. From diverse agent runs, we generate a broad set of policies and create both within subdomain and cross subdomain pairings with violation labels. In addition to full-trajectory evaluation, PolicyGuardBench also includes a prefix-based violation detection task where models must anticipate policy violations from truncated trajectory prefixes rather than complete sequences. Using this dataset, we train PolicyGuard-4B, a lightweight guardrail model that delivers strong detection accuracy across all tasks while keeping inference efficient.
Notably, PolicyGuard-4B generalizes across domains and preserves high accuracy on unseen settings.
Together, PolicyGuardBench and PolicyGuard-4B provide the first comprehensive framework for studying policy compliance in web agent trajectories, and show that accurate and generalizable guardrails are feasible at small scales.

Primary Area: alignment, fairness, safety, privacy, and societal considerations

Submission Number: 10489