Dual Learning Model of Code Summary and Generation Based on Transformer

Jiaying Wang, Lijun Cao, Jing Shan, Xiaoxu Song, Junyi Jiang

Abstract: Code summary (CS) produces natural language descriptions based on code snippets, while code generation (CG) produces code snippets based on natural language. Since both tasks are intended to model the relationship between natural language and code snippets, recent research has combined these tasks to improve their performance. The existing approach either relies on LSTM for dual training, which makes it impossible to address the issue of long-distance dependency or has an imbalance in the performance between code generation and code summary. In this paper, an end-to-end model based on Transformer is proposed to handle these problems. We propose two new regularization terms to not only constrain the duality of the two models by explicitly utilizing the probability correlation between CS and CG but also promote alignment between CS and CG models. Based on this, we propose a dual-learning algorithm for CS and CG. Experiments on real Java and Python datasets demonstrated that our model significantly improved the results of CS and CG tasks, surpassing the performance of existing models.