SoK: Large Language Models in Security Code Review and Testing
Keywords: LLMs, software security, security testing, vulnerability detection, fuzz testing
TL;DR: This is a SoK paper presenting and discussing practical applications of LLMs in software security, specifically in code vulnerability detection, fuzz testing and exploit generation.
Abstract: In this paper, we present and discuss practical applications of large language models (LLMs) in software security, concretely in code vulnerability detection, fuzz testing and exploit generation. Measurements of various research outcomes are analysed to answer questions about the performance of LLM in those fields, including a comparison with tools following traditional approaches. In addition, the drawbacks and a future overlook with a delineation of technical challenges are given. Challenges are found in the cost- and time-intensive training of LLM, the limited context-length understanding of program code, the high false positive rate because of hallucinations, and keeping the data up-to-date so that definitions of newly detected vulnerabilities are contained.
Area: System Security
Type: Systemization of Knowledge (SoK)
Conflicts: All(Zurich University of Applied Sciences)
Potential Reviewers: Stefan Brunthaler, Amir H. Payberah
Revision: No
Contact Email: beljuedi@students.zhaw.ch, gueu@zhaw.ch
Submission Number: 2
Loading