Transformer-XL With Graph Neural Network for Source Code Summarization

Xiaoling Zhang, Shouguo Yang, Luqian Duan, Zhe Lang, Zhiqiang Shi, Limin Sun

Abstract: Source code summarization is the task of generating a readable natural language to describe the functionality of source code. Code summarization is rapidly expanding, especially as the research takes great advantage of advances in neural networks and artificial intelligence technologies. Some mainstream methods input the structural information (abstract syntax tree (AST)) of the source code into the language model to generate relatively satisfactory comments. However, existing methods can not capture code’s long dependencies from AST for effective code summarization. In this paper, we provide a novel way to generate code summaries by combining a graph-based neural network and a Transformer-XL network. We utilize the graph-based neural network to better capture the structure information of AST, and the Transformer-XL network to learn important tokens in the AST and alleviate the problem of long dependency. We evaluate our technique on the standard Java dataset. The experimental results show that the effectiveness of our model is remarkable. It pushes the precision score to 60.73% (5.21% absolute improvement) and the F1 score to 51.06%.