Enhancing Tool Calling in LLMs with the International Tool Calling (ITC) Dataset

ACL ARR 2025 May Submission5656 Authors

Abstract: Tool calling allows large language models (LLMs) to interact with external systems like APIs, enabling applications in customer support, data analysis, and dynamic content generation. Despite recent advances, challenges persist due to limited datasets with simulated or inaccessible APIs and insufficient geographic diversity. To address this, we present the International Tool Calling (ITC) dataset, designed for real-world, international tool-calling scenarios. ITC includes 3,571 real APIs and 17,540 tool-calling tasks across 20 categories and 40 countries. The dataset was constructed through a four-stage pipeline: API collection and construction, query generation, query scoring and filtering, and question–answer pair generation. Experiments reveal substantial performance gaps between open- and closed-source LLMs, while fine-tuning on ITC significantly improves generalization. ITC offers a valuable resource for advancing LLM capabilities in complex, multi-tool, and international contexts. Dataset: \href{https://anonymous.4open.science/r/International-Tool-Calling-ITC-dataset-5FD7/}{https://anonymous.4open.science/r/International-Tool-Calling-ITC-dataset-5FD7/}.

Paper Type: Long

Research Area: Resources and Evaluation

Research Area Keywords: corpus creation, benchmarking, language resources, multilingual corpora, NLP datasets, evaluation, datasets for low resource languages

Contribution Types: Publicly available software and/or pre-trained models, Data resources, Data analysis

Languages Studied: English, Chinese, Japanese

Submission Number: 5656