A Road for LLM SQL Bug-Fixing Enhancing
Abstract: Code Large Language Models (Code LLMs), such as Code llama and DeepSeek-Coder, have demonstrated exceptional performance in the code generation tasks. However, most existing models focus on the abilities of generating correct code, but weak in bug code repair.
In this paper, we introduce a suit of methods which enhance LLM's bug-fixing abilities on SQL code, which are mainly consisted of two parts: A Progressive Dataset Construction (PDC) from scratch and Dynamic Mask Supervised Fine-tuning (DM-SFT).
PDC proposes two data expansion methods from the perspectives of breadth first and depth first respectively. DM-SFT introduces an efficient bug-fixing supervised learning approach, which effectively reduce the total training steps and mitigate the "mental disorientation" in SQL code bug-fixing training. In our evaluation, the code LLM models trained on these two methods have exceeds all current best performing model which size is much larger.
Paper Type: Long
Research Area: Generation
Research Area Keywords: efficient models, argument mining
Contribution Types: Model analysis & interpretability, Approaches to low-resource settings, Approaches low compute settings-efficiency
Languages Studied: SQL, English
Submission Number: 461
Loading