Explain-then-translate: an analysis on improving program translation with self-generated explanations

Zilu Tang, Mayank Agarwal, Alexander G Shypula, Bailin Wang, Derry Wijaya, Jie Chen, Yoon Kim

Submission Type: Regular Long Paper

Submission Track: Natural Language Generation

Submission Track 2: Machine Translation

Keywords: code generation, machine translation, program translation, large language model, prompting, chain-of-thought

TL;DR: We analyzed how useful self-generated natural language explanations of code can be in the setting of program translations

Abstract: This work explores the use of self-generated natural language explanations as an intermediate step for code-to-code translation with language models. Across three types of explanations and 19 programming languages constructed from the MultiPL-E dataset, we find the explanations to be particularly effective in the zero-shot case, improving performance by 12\% on average. Improvements with natural language explanations are particularly pronounced on difficult programs. We release our dataset, code, and canonical solutions in all 19 languages.

Submission Number: 196