Understanding Secret Leakage Risks in Code LLMs: A Tokenization Perspective

Meifang Chen, Zhe YANG, HUANG Nianchen, Yizhan Huang, Yichen LI, Michael R. Lyu

Keywords: Tokenization, Code LLM, Secret Leakage

Abstract: Code secrets are sensitive assets for software developers, while their leakage imposes high risks to cybersecurity. While the rapid development of AI code assistants powered by Code Large Language Models (CLLMs) revolutionizes the landscape of software engineering, CLLMs are shown to inadvertently leak such secrets due to a notorious memorization phenomenon. This study firstly reveals that Byte-Pair Encoding (BPE) tokenization leads to unexpected behaviour of secret memorization. Specifically, we discovered that some secrets are among the easiest for CLLMs to memorize. These secrets exhibit high character-level entropy, but low token-level entropy. We name it as gibberish bias. We identified the root of the bias as the token distribution shift between the CLLM training data and the secret data. We further discuss how gibberish bias manifests under the "larger vocabulary" trend. To conclude the paper, we discuss potential mitigation strategies and the broader implications for the current tokenizer design.

Submission Number: 17