Context manipulation attacks : Web agents are susceptible to corrupted memory

Atharv Singh Patlan, S Ashwin Hebbar, Pramod Viswanath, Prateek Mittal

Submission Track: Paper Track (up to 8 pages)

Keywords: Computer Use Agents, Web Browsing Agents, Memory Injection, Attacks, Prompt Injection

TL;DR: CUAs, specifically web-agents, are vulnerable to manipulated context - we demonstrate this via "plan injection"

Abstract:

Autonomous web navigation agents, which translate natural language instructions into sequences of browser actions, are increasingly deployed for complex tasks spanning e-commerce, information retrieval, and content discovery. Due to the stateless nature of large language models (LLMs), these agents rely heavily on external memory systems to maintain context across interactions. Unlike centralized systems where context is securely stored server-side, agent memory is often managed client-side or by third-party applications, creating significant security vulnerabilities - this was recently exploited to attack production systems.

We introduce and formalize "plan injection," a novel memory manipulation attack that corrupts these agents' internal task representations by targeting this vulnerable context. Our systematic evaluation across two agent architectures, Browser-use: a simple browser automation framework and Agent-E : a hierarchical planner-executor system, reveals that even agents with robust prompt injection defenses remain susceptible to memory manipulation. We demonstrate that contextually-aligned malicious instructions achieve up to 3× higher success rates than context-agnostic insertions. Further, we show that "context-chained injections", a injection crafted to create logical bridges between legitimate user tasks and attacker objectives, leads to a 17.7% increase attack success rate for privacy exfiltration tasks. These findings highlight the importance of secure memory design in autonomous web agents beyond existing prompt-based defenses.

Submission Number: 44