BlockFound: Customized blockchain foundation model for anomaly detection

Jiahao Yu, Xian Wu, Hao Liu, Wenbo Guo, Xinyu Xing

Keywords: Large language model, blockchain, anomaly detection

TL;DR: We propose BlockFound, a transformer-based model for detecting anomalous DeFi transactions, which outperforms existing methods through advanced tokenization and masked language modeling.

Abstract:

We propose BlockFound, a customized foundation model for anomaly blockchain transaction detection.
Unlike existing methods that rely on rule-based systems or directly apply off-the-shelf large language models, BlockFound introduces a series of customized designs to model the unique data structure of blockchain transactions.
First, a blockchain transaction is multi-modal, containing blockchain-specific tokens, texts, and numbers.
We design a modularized tokenizer to handle these multi-modal inputs, balancing the information across different modalities.
Second, we design a customized mask language learning mechanism for pretraining with RoPE embedding and FlashAttention for handling longer sequences.
After training the foundation model, we further design a novel detection method for anomaly detection.
Extensive evaluations on Ethereum and Solana transactions demonstrate BlockFound's exceptional capability in anomaly detection while maintaining a low false positive rate.
Remarkably, BlockFound is the only method that successfully detects anomalous transactions on Solana with high accuracy, whereas all other approaches achieved very low or zero detection recall scores.
This work not only provides new foundation models for blockchain but also sets a new benchmark for applying LLMs in blockchain data.

Primary Area: foundation or frontier models, including LLMs

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 8906