StruPhantom: Hijacking Black-Box LLM Tabular Agents via Structure-Aware Reasoning

ACL ARR 2026 January Submission3335 Authors

Keywords: Indirect Prompt Injection, Language Agents, Tool-Use Agents, Large Language Model, AI Security

Abstract: LLM-powered autonomous agents that process tabular data (tabular agents) are increasingly deployed in business-critical applications. While LLMs are known to be vulnerable to prompt injection attacks, tabular agents enforce strict data formats and structural rules, making conventional attacks largely ineffective. To address this, we propose StruPhantom, a novel attack that targets black-box tabular agents by automatically generating structure-aware malicious inputs. StruPhantom uses a constrained Monte Carlo Tree Search augmented with an off-topic evaluator to iteratively refine attack templates, effectively navigating multi-layered structural data to achieve goal hijacking. Our evaluation across multiple LLM-based agents, including real-world platforms, shows that StruPhantom outperforms prior methods by over 50% in attack success rate, reliably inducing outputs containing phishing links or malicious code.

Paper Type: Long

Research Area: Safety and Alignment in LLMs

Research Area Keywords: LLM/AI agents,safety and alignment,security and privacy

Contribution Types: Model analysis & interpretability

Languages Studied: English

Submission Number: 3335