Topology of Attention Detects Hallucinations in Code LLMs

ACL ARR 2026 January Submission10393 Authors

Keywords: Code Models, Robustness, Attention Matrices

Abstract:

While the AI-code assistant tools become widespread, automatic assessment of the correctness of the generated code becomes a significant challenge. Code LLMs are prone to hallucinations, which may lead to code that does not solve a required problem, or even to code with severe security vulnerabilities.
In this paper, we propose a new approach to assessment of code correctness. Our solution is based on topological data analysis (TDA) of attention maps of code LLMs.
We carry out experiments with common benchmarks (HumanEval, MBPP, MultiPL-E), 5 programming languages and 10 code LLMs of size up to 34B parameters.
The experimental results show that the proposed method is better than recent baselines. Moreover, the trained classifiers are transferable between coding benchmarks.

Paper Type: Long

Research Area: Code Models

Research Area Keywords: Code language models, evaluation of code models, safety and reliability of code models

Languages Studied: Python, Java, Go, Rust, Lua

Submission Number: 10393