Functional consistency of LLM code embeddings: A self-evolving data synthesis framework for benchmarking

Zhuohao Li, Wenqing Chen, Jianxing Yu, Zhichao Lu

Abstract: Highlights•We propose a novel Functionality-Oriented Code Self-Evolution framework that systematically generates diverse code variants, effectively covering different syntactic and semantic consistency scenarios.•All generated code samples are accompanied by rigorous test cases to verify their functional correctness, ensuring high dataset quality and reliability.•Extensive experiments on multiple downstream tasks–including code clone detection, functional consistency identification, and code retrieval–demonstrate significant improvements when models are trained on the evolved datasets.•Our study reveals inherent limitations of existing code embedding models in capturing functional semantics and shows that training with self-evolved data substantially enhances their ability to understand and generalize functional code equivalences.

External IDs:dblp:journals/eswa/LiCYL26