Agent4Vul: multimodal LLM agents for smart contract vulnerability detection

Smart contract vulnerabilities have emerged as a significant threat to blockchain system security under the Web 3.0 ecosystem. According to recent research, large language models (LLMs) have demonstrated immense potential in smart contract security audits but still lack the capability for effective vulnerability detection. Consequently, leveraging the capabilities of LLMs to effectively enhance the performance of smart contract vulnerability detection remains a critical challenge. In this paper, we propose Agent4Vul, a novel framework utilizing multimodal LLM agents to enhance smart contract vulnerability detection. Specifically, we design two LLM-based agents: Commentator and Vectorizer. The Commentator agent generates comments for the source code, while the Vectorizer agent converts contents into vector representations. Subsequently, we develop a multimodal learning architecture comprising the semantic branch and the graph branch, which collectively integrate features from the source code, generated comments, and the bytecode control flow graph (CFG). We empirically evaluate a large-scale real dataset of smart contracts and compare 19 state-of-the-art baseline approaches. The results show that Agent4Vul achieves (1) superior performance over all baseline approaches on the four types of common vulnerabilities in real attacks; (2) 3.61%–16.32% higher F1-scores than existing artificial intelligence (AI) approaches, outperforming even advanced LLMs like GPT-4o and o1. This work lays a solid foundation for LLM-driven smart contract security and introduces innovative applications of LLMs in software engineering.