ToolCoder: Enabling Code Generation Models to Use Unknown APIs with API Search Tools

Anonymous

Abstract: Automatically generating source code from natural language descriptions has been a growing field of research in recent years. Invoking correct APIs is crucial to code generation. However, existing code generation models struggle in handling unknown APIs (e.g., user-private projects and libraries) , often generating erroneous or even non-existent APIs. Inspired by the process of human developers using code search tools to learn unknown APIs, we propose ToolCoder, a novel approach that integrates API search tools with existing models to assist code generation and API selection. ToolCoder automatically invokes API search tools to retrieve relevant APIs and learns API usages from retrieved results. Our experimental results demonstrate that ToolCoder exhibits excellent performance and generalization ability across five public and private library code generation benchmarks, with at least 6.21% improvement on average pass@1 metrics and 9.64% improvement on average pass@10 metrics compared to state-of-the-art methods. Furthermore, we show that our relatively small ToolCoder model is comparable to one of the current best models, i.e.,, GPT-3.5, highlighting the potential of incorporating programming tools into the code generation process.

Paper Type: long

Research Area: NLP Applications

Contribution Types: NLP engineering experiment

Languages Studied: English