Alternate Task Technique: Improving Natural Language to Code in Low-Resource Languages

Anonymous

TL;DR: We introduce ``alternate task technique'' that uses an LLM to perform a surrogate task whose results are combined with the LLM's results on the original task to improve performance on the original NL2Code task

Abstract: When using large language models (LLMs) to generate code from natural language (NL2Code), the target programming language influences precision. We empirically observe that LLMs are more likely to produce correct code when the target language is a popular language and struggle for low-resource target programming languages. Prompt engineering approaches can address the problem to some extent but cannot fully close the gap between popular and low-resource target languages. We introduce "alternate task technique" that uses an LLM to perform a surrogate task whose results are combined with the LLM's results on the original task to improve performance on the original NL2Code task. Using SQL, Python Pandas, and Power Query language M as three targets, we show that our approach brings the performance of LLMs on the low-resource M language significantly closer to its performance on the more popular Python Pandas and SQL languages.

Paper Type: long

Research Area: NLP Applications

Contribution Types: Model analysis & interpretability, Approaches to low-resource settings

Languages Studied: English