TreeBERT: A tree-based pre-trained model for programming language

Xue Jiang, Zhuoran Zheng, Chen Lyu, Liang Li, Lei Lyu

Abstract: Source code can be parsed into the abstract syntax tree (AST) based on defined syntax rules. However, in pre-training, little work has considered the incorporation of tree structure into the learni...