Alpha-SQL: Zero-Shot Text-to-SQL using Monte Carlo Tree Search

Boyan Li, Jiayi Zhang, Ju Fan, Yanwei Xu, Chong Chen, Nan Tang, Yuyu Luo

Abstract:

Text-to-SQL, which enables natural language interaction with databases, serves as a pivotal method across diverse industries.
With new, more powerful large language models (LLMs) emerging every few months, fine-tuning has become incredibly costly, labor-intensive, and error-prone. As an alternative, *zero-shot* Text-to-SQL, which leverages the growing knowledge and reasoning capabilities encoded in LLMs without task-specific fine-tuning, presents a promising and more challenging direction.
To address this challenge, we propose Alpha-SQL, a novel approach that leverages a Monte Carlo Tree Search (MCTS) framework to iteratively infer SQL construction actions based on partial reasoning states. To enhance the framework’s reasoning capabilities, we introduce *LLM-as-Action-Model* to dynamically generate SQL construction *actions* during the MCTS process, steering the search toward more promising SQL queries. Moreover, Alpha-SQL employs a self-supervised reward function to evaluate the quality of candidate SQL queries, ensuring more accurate and efficient query generation. Experimental results show that Alpha-SQL achieves 69.7% execution accuracy on the BIRD development set, using a 32B open-source LLM without fine-tuning. Alpha-SQL outperforms the best previous zero-shot approach based on GPT-4o by 2.5% on the BIRD development set.

Lay Summary:

Asking databases complex questions using natural language is challenging because translating them into precise database code (SQL) is very difficult. While powerful AI models (LLMs) can help, repeatedly retraining them for every new database or model is costly and time-consuming. This hinders the efficient application of the latest AI technology in database interaction.

We developed Alpha-SQL, a new method that allows AI to translate natural language into SQL without expensive retraining. Alpha-SQL acts like a smart planner, guiding the AI to build SQL code step-by-step. It explores various paths for constructing the code (like navigating a maze) and uses the AI's own reasoning to decide the best "next move". It also cleverly self-checks by comparing the execution results of different attempts.

Alpha-SQL enables smaller, more open AI models to achieve high accuracy in this complex task. For example, using our approach, a 32-billion-parameter open-source AI outperformed the much larger GPT-4o system on a key industry benchmark. This makes advanced natural language database interaction more accessible, cost-effective, and easier to deploy, helping more people gain insights from their data.

Application-Driven Machine Learning: This submission is on Application-Driven Machine Learning.

Link To Code: https://github.com/HKUSTDial/Alpha-SQL

Primary Area: Applications->Everything Else

Keywords: Text-to-SQL, Databases, LLMs

Submission Number: 9980