TRADERBENCH: HOW ROBUST ARE AI AGENTS IN ADVERSARIAL CAPITAL MARKETS?

Xiaochuang Yuan, Hui Xu, Silvia Xu, Cui Zou, Jing Xiong

Keywords: AgentBeats, Financial Benchmarking, AI Financial Agents, Adversarial Trading, Tool Use, Conceptual-vs-Computational Gap

Abstract:

Evaluating AI agents in finance faces two key challenges: static benchmarks require costly expert annotation yet miss the dynamic decision-making central to real-world trading, while LLM-based judges introduce uncontrolled variance on domain-specific tasks. We introduce TraderBench, a benchmark that addresses both issues. It combines expert-verified static tasks (knowledge retrieval, analytical reasoning) with adversarial trading simulations scored purely on realized performance---Sharpe ratio, returns, and drawdown---eliminating judge variance entirely. The framework features two novel tracks: crypto trading with four progressive market-manipulation transforms, and options derivatives scoring across P\&L accuracy, Greeks, and risk management. Trading scenarios can be refreshed with new market data to prevent benchmark contamination. Evaluating 12 models (8B open-source to frontier) on ${\sim}$50 tasks, we find:
(1) 7 of 12 models score ${\sim}$33 on crypto with $<$1-point variation across adversarial conditions, exposing fixed non-adaptive strategies;
(2) extended thinking helps retrieval (+26 points) but has zero impact on trading (+0.3 crypto, $-$0.1 options).
These findings reveal that current agents lack genuine market adaptation, underscoring the need for performance-grounded evaluation in finance.

Email Sharing: We authorize the sharing of all author emails with Program Chairs.

Data Release: We authorize the release of our submission and author names to the public in the event of acceptance.

Submission Number: 193