Agentic Context Strategies for Multi-Format Document Understanding: When Should Language Models Use Tools?

Mukul Singh, Mansi Uniyal, Ryan Nadel

Keywords: Tool-augmented LLMs, Retrieval-Augmented Generation, Document QA, Agentic Systems, Long-Context Reasoning

TL;DR: In enterprise document QA, longer context is not enough. Across 5,400 runs and three document formats, we show that tool-augmented LLMs outperform passive RAG by up to 7.7×. Intelligent routing—not bigger models or longer context—drives performance.

Abstract: Large language models face fundamental trade-offs when processing long documents: full context is expensive and may exceed limits, while RAG risks missing relevant information. We evaluate four context strategies across six frontier models on three document formats (Word, Excel, and PowerPoint). Our key finding: agentic tool-augmented approaches dramatically outperform passive strategies, with RAG+Tools achieving 46\% accuracy vs 6\% for RAG-only. Tool benefits are consistent across formats (+28-40 points) and models. We further show that (1) intelligent routing matters more than iteration count, (2) tools provide unique capability beyond reasoning loops, and (3) forcing active exploration matches providing context proactively. These results suggest tool augmentation is crucial for complex document QA.

Submission Type: Discovery

Submission Number: 463