SkyRL-SQL: Multi-turn SQL Data Agents via RL

Shu Liu, Alan Zhu, Sumanth Hegde, Shiyi Cao, Shuo Yuan, Samion Suwito, Tyler Griggs, Matei Zaharia, Joseph E. Gonzalez, Ion Stoica

Keywords: text-to-sql, multiturn RL, LLM agents

Abstract: Recent advances in reinforcement learning (RL) have enabled large language models (LLMs) to act as interactive agents across domains like coding and web search. However, database interactions, particularly in the Text-to-SQL domain, remain underexplored. Existing work largely treats Text-to-SQL as a one-shot generation task, which struggles in real-world scenarios where natural language queries are often ambiguous and databases are messy. We introduce a simple and data-efficient multi-turn RL approach that enables LLMs to probe databases, refine queries, and verify results across successive interactions, mirroring human exploratory analysis. Using only about 600 training examples, our 7B-parameter model improves execution accuracy by up to 9.2\% on Spider benchmarks, outperforming stronger baselines including o4-mini, GPT-4o, and open-source SFT models trained on orders of magnitude more data. Our results highlight the promise of multi-turn RL as a scalable, practical approach for building robust Text-to-SQL agents.

Submission Number: 102