OctoTools: An Agentic Framework with Extensible Tools for Complex Reasoning

Pan Lu, Bowen Chen, Sheng Liu, Rahul Thapa, Joseph Boen, James Zou

Keywords: Foundation Models, Agentic Systems, Complex Reasoning

TL;DR: We introduce OctoTools, a training-free, user-friendly, and extensible agentic framework designed to tackle complex reasoning across diverse domains.

Abstract: Solving complex reasoning tasks involve visual understanding, domain knowledge retrieval, numerical calculation, and multi-step reasoning. Existing methods augment foundation models (FMs) with external tools but are restricted to specialized domains, limited tool types, or require additional training data. In this paper, we introduce OctoTools, a training-free, user-friendly, and easily extensible open-source agentic framework designed to tackle complex reasoning across diverse domains. OctoTools introduces standardized tool cards to encapsulate tool functionality, a planner for both high-level and low-level planning, and an executor to carry out tool usage. We validate OctoTools’ generality across 16 diverse tasks (including MathVista, MMLU-Pro, MedQA, and GAIA-Text), achieving substantial average accuracy gains of 9.3\% over GPT-4o. Furthermore, OctoTools also outperforms AutoGen, GPT-Functions and LangChain by up to 10.6\% when given the same set of tools. Through comprehensive analysis and ablations, OctoTools demonstrates advantages in task planning, effective tool usage, and multi-step problem solving. Code and interactive demos are available at https://octotools.github.io/.

Submission Number: 43