Smart Contract Detection with LLM Achieves 0.9223 Accuracy, Addressing Uncertainty for EVM Bytecode Analysis

Detecting malicious smart contracts before deployment represents a crucial step in protecting against financial losses on blockchain platforms like Ethereum and BSC. Yating Liu, Xing Su, Hao Wu, and colleagues from Nanjing University, along with Yuxi Cheng and Fengyuan Xu, present a new framework, FinDet, that significantly improves the detection of these adversarial contracts. The team addresses key limitations of existing methods and large language models (LLMs), specifically the difficulty LLMs have processing complex compiled code and providing confident binary classifications. FinDet extracts semantic information from the low-level code, enabling the LLM to reason more effectively, and also assesses the LLM’s certainty in its answers, improving reliability. Comprehensive evaluation demonstrates FinDet achieves a balanced accuracy of 0. 9223 and a true positive rate of 0. 8950, outperforming current methods and maintaining robustness against evolving attack strategies, even identifying previously unreported malicious contracts in real-world testing.

Current approaches often rely on superficial pattern matching, failing to grasp the underlying semantic intent of the code and resulting in inaccurate assessments. FinDet employs a holistic, multi-view behavioral semantic analysis, considering the overall behavior and meaning of the contract rather than simply examining individual lines of code. This system analyzes contracts from multiple perspectives, including function-level, fund-flow level, and contract-level, to gain a comprehensive understanding of its operation. FinDet utilizes confidence-based probing and fusion, employing multiple prompts and scoring mechanisms to assess the likelihood of a contract being adversarial and combining these scores for a final prediction.

Bytecode Lifting for Smart Contract Analysis

Scientists developed FinDet, a novel framework for detecting adversarial smart contracts before deployment, overcoming the limitations of rule-based and machine learning methods. Recognizing that existing approaches struggle with unseen attack patterns and require substantial labeled data, the team harnessed the potential of large language models (LLMs) while addressing the challenge of interpreting low-level code. The research pioneered a two-stage process beginning with bytecode lifting, transforming complex EVM bytecode into a semi-structured natural language description to facilitate LLM analysis. To address LLM input limitations, scientists condensed the bytecode representation, preserving crucial behavioral information within typical LLM constraints.

The team then enhanced the LLM’s ability to assess its own certainty, developing a technique to probe and measure uncertainty through multi-round questioning, improving the robustness of binary classifications. Experiments demonstrate that FinDet achieves a balanced accuracy of 0. 9223 and a true positive rate of 0. 8950, significantly outperforming baseline approaches and successfully detecting previously unreported adversarial contracts.

FinDet Detects Adversarial Contracts From Bytecode

FinDet represents a breakthrough in detecting adversarial smart contracts directly from their compiled bytecode, achieving a balanced accuracy of 0. 9223 and significantly outperforming existing methods. This enables proactive identification of malicious contracts before deployment, preventing potential financial losses and representing a substantial advancement in blockchain security. FinDet leverages the reasoning capabilities of large language models (LLMs) while addressing key challenges in applying these models to low-level code analysis. The system accurately identifies adversarial contracts by first translating the raw bytecode into a semi-structured natural language description, enhancing interpretability and enabling more accurate detection of malicious behaviors.

A core achievement is the development of a method to assess the uncertainty of LLM responses, crucial for reliable binary classifications. Researchers transformed the detection query into a task of assigning probabilities across four levels of uncertainty, repeating the task with varied prompts to gauge consistency. By measuring the entropy of these probability distributions, the team derived a high-confidence answer, improving the system’s robustness. During a ten-day real-world test, FinDet successfully identified 30 adversarial contracts, including 25 previously unreported instances, and the team curated a comprehensive dataset for the blockchain security community.

FinDet Detects Adversarial Contracts with High Accuracy

FinDet offers a new approach to detecting adversarial smart contracts by directly analyzing EVM bytecode. The framework enhances semantic understanding by translating low-level code into a more easily interpretable natural language format, enabling a multi-faceted analysis of contract behavior. A key innovation lies in the incorporation of fund-flow reachability analysis, which strengthens the precision of semantic interpretation and captures the distinct stages of malicious attacks. Evaluations demonstrate a balanced accuracy of 0. 9223 and a true positive rate of 0.

8950, significantly exceeding the performance of existing methods. The system exhibits robust generalization to previously unseen attack patterns and maintains effectiveness even with limited training data or when faced with obfuscated code. This research offers a practical advancement in securing blockchain ecosystems against emerging threats by providing a proactive and effective means of identifying malicious smart contracts.