HLR-SQL: Human-Like Reasoning for Text-to-SQL

Timo Eckmann, Matthias Urban, Jan-Micha Bodensohn, Carsten Binnig

Keywords: Human-Like Reasoning, Text-to-SQL, Iterative Query Refinement, Complex SQL Queries, Spider-HJ, LLMs

TL;DR: We propose an autonomous Text-to-SQL prompting technique that iteratively builds queries with many joins.

Abstract:

Recent LLM-based approaches have achieved impressive results on
Text-to-SQL benchmarks such as Spider and Bird. However, a key
limitation of these benchmarks is that their queries do not reflect
the complexity typically seen in real-world enterprise scenarios. In
this paper, we introduce HLR-SQL, a new approach designed to han-
dle such complex enterprise SQL queries. Unlike existing methods,
HLR-SQL imitates Human-L ike Reasoning with LLMs by incremen-
tally composing queries through a sequence of intermediate steps,
gradually building up to the full query. We evaluate HLR-SQL on
a newly constructed benchmark, Spider-HJ, which systematically
increases query complexity by splitting tables in the original Spi-
der dataset to raise the average join count needed by queries. Our
experiments show that state-of-the-art models experience up to
a 70% drop in execution accuracy on Spider-HJ, while HLR-SQL
achieves a 9.51% improvement over the best existing approaches
on the Spider leaderboard.

Submission Number: 3