LLMSmartSec: Smart Contract Security Auditing with LLM and Annotated Control Flow Graph

Viraaji Mothukuri, Reza M. Parizi, James L. Massa

Abstract: Historically, the complexity of identifying vulnerabilities in smart contracts required human-intensive audits to supplement imprecise automated code scans. The growing smart contract market highlights the urgency for effective automated security auditing. Furthermore, new AI-powered coding assistants can introduce vulnerabilities that evade engineers, and autonomous AI can now write 100% code changes without human oversight. This research introduces LLMSmartSec, a new approach that accurately identifies and fixes smart contract vulnerabilities, leveraging the strengths of machine intelligence to operate at high speed, scale, and without fatigue. To develop LLMSmartSec, we first fine-tuned Open AI GPT-4 to understand Solidity, the programming language of Ethereum blockchain, and to micro-analyze smart contracts holistically from the viewpoints of the developer (LLMDev), auditor (LLMAudit), and ethical hacker (LLMeHack) and to identify vulnerabilities and generate code fixes for them. We used GPT-4 again to store the smart contract code in a Control Flow Graph (CFG) annotated with the code vulnerabilities and their fixes. Finally, we trained an LLMGraphAgent with open-source LLMs on the annotated CFG so that it can run locally, identifying vulnerabilities and fixes without costly calls to GPT-4. This research enhances the security of blockchain projects in Web3 by supplementing a costly human bottleneck with a low-cost automated security tool based upon the innovative creation of an annotated CFG with GPT-4.