CREDIT-SQL: Few-shot prompting for context-dependent text-to-SQL with regularized examples from diversity sampling

Anonymous

TL;DR: We propose a few-shot prompting method called CREDIT-SQL for the context-dependent text-to-SQL problem, which utilizes the question rephrasing and a systematic selection of in-context examples.

Abstract: In this paper, we propose a few-shot prompting method called CREDIT-SQL for the context-dependent text-to-SQL problem. CREDIT-SQL converts each question in a multi-turn dialogue into a self-contained question with a fixed few-shot prompt. Once a self-contained question is obtained, CREDIT-SQL converts it into an SQL query using a prompt made of in-context examples selected by diversity sampling and subsequent example voting. CREDIT-SQL with ChatGPT 3.5 achieves 58.6\% in terms of the exact set match without values on the dev set of CoSQL, which is the performance comparable to the state-of-the-art models for context-dependent text-to-SQL. We also argue that the example voting we introduced in CREDIT-SQL can serve as an efficient and effective way to mitigate the instability of in-context example selection in general.

Paper Type: long

Research Area: Question Answering

Contribution Types: NLP engineering experiment

Languages Studied: English