A transformer-based framework for software vulnerability detection using attention-driven convolutional neural networks

Open Webpage

Abdelkarim Smaili, Yanan Zhang, Mekkaoui Djamel Eddine, Mohamed Amine Midoun, Mohamed Zakariya Talhaoui, Hamidaoui Meryem, Weiqiang Kong

Published: 2025, Last Modified: 06 Nov 2025Eng. Appl. Artif. Intell. 2025EveryoneRevisionsBibTeXCC BY-SA 4.0
Abstract: In the realm of software systems and quality assurance, vulnerability identification has become a critical concern in today’s connected world. Vulnerabilities not only make systems less effective, but they also pose significant risks to user privacy and data integrity. Although automated vulnerability detection has seen considerable progress, existing approaches frequently struggle to accurately model syntactic and semantic code relationships. This deficiency leads to undetected vulnerabilities and false positives, thereby undermining the efficacy of software security protocols. To overcome these limitations, we propose a new vulnerability detection technique called CodeGATNet (Code-Gated Attention using Convolutional Features). Our deep learning model is designed to handle static vulnerability detection in source code to enable batch-level examination of codebases. CodeGATNet operates in two phases: Static Code Embedding Generation (SCEG) and Convolutional Attention Network for Feature Refinement (CAN-FR). SCEG utilizes fine-tuned CodeBERT embeddings (a pretrained transformer model for programming languages), which are task-specifically optimized to improve the pretrained model’s capacity to capture application-relevant semantic patterns while maintaining its general code comprehension capabilities. CAN-FR uses a hybrid architecture for feature extraction and refinement combining a gated attention mechanism with one-Dimensional Convolutional Neural Network (1D-CNN). This hybrid approach helps the model to efficiently capture present global contextual relationships in the source code as well as local structural patterns. Extensive experimental evaluations on three large-scale (C/C++) real-world datasets and the results show that CodeGATNet considerably outperforms leading models, obtaining accuracy enhancements of 18.05%, 28.14%, and 13.06%, alongside F1-score improvements of 7.83%, 18.28%, and 13.0%.