GEAR: Training-Free Rule Distillation for Advanced and Efficient Tool-Augmented Reasoning

ACL ARR 2026 January Submission6602 Authors

Keywords: AI Agents, Tool Use, Training free, Distillation, Reasoning, Efficiency

Abstract: Tool use enables AI agents to interact with external systems and accomplish real-world tasks beyond text generation. Large reasoning models (LRMs) excel at tool use but rely on test-time compute that dramatically increases inference latency and cost. Distilling LRM outputs to smaller models has emerged as an efficient alternative. However, most researchers and developers face resource constraints with only API access and no training compute, precluding distillation. We introduce $\mathbf{GEAR}$, a training-free framework that extracts compositional rules from tool-use failures through meta-reasoning and distills them into inference prompts to guide model reasoning. We hypothesize that combining explicit rules with examples provides richer guidance than either alone. Across 78 API domains, $\mathbf{GEAR}$ achieves comparable performance to 3-shot prompting, while $\mathbf{GEAR}$ with only 2 examples outperforms 3-shot by 9.7\% on the leading proprietary LRM (Claude-4.5-Sonnet), validating our complementarity hypothesis. Remarkably, $\mathbf{GEAR}$ enables Qwen3-30B to surpass the best Claude configuration by 15.7\% while being 5$\times$ faster, demonstrating training-free democratization of tool-use capabilities.

Paper Type: Long

Research Area: AI/LLM Agents

Research Area Keywords: LLM agents, tool use, planning in agents, other LLM agent topics

Contribution Types: NLP engineering experiment, Approaches low compute settings-efficiency

Languages Studied: English

Submission Number: 6602