Abstract: Highlights•Natural language knowledge is used in the program representation learning.•A novel model PM2-CNN is proposed for software defect prediction.•PM2-CNN is based on Transformer architecture and multi-channel CNN.•Pre-trained language model is introduced to obtain context-sensitive representations.•The validity of the proposed method is verified by using Big-Vul dataset.
Loading