OD-Stega: LLM-Based Near-Imperceptible Steganography via Optimized Distributions

Yu-Shin Huang, Peter Just, Krishna Narayanan, Chao Tian

Keywords: LLLM, steganography, arithmetic coding

Abstract: We consider coverless steganography where a Large Language Model (LLM) drives an arithmetic coding decoder to generate stego-text. An efficient method should embed secret message bits in as few language tokens as possible, while still keeping the stego-text natural and fluent. We show that on the individual token level, this problem is mathematically equivalent to maximizing the entropy of a replacement probability distribution of the next token generation, subject to a constraint on the KL divergence between the chosen probability distribution and the original distribution given by the LLM. A closed-form solution is provided for the optimization problem, which can be computed efficiently. Several important practical issues are also tackled: 1) The combination of the optimized distribution and the vocabulary truncating technique is considered, 2) An often-overlooked tokenization mismatch issue is resolved with a simple prompt selection approach, and 3) The combination of the optimized distribution with other sequence-level selection heuristics to further enhance the efficiency and reliability is studied.

Primary Area: applications to computer vision, audio, language, and other modalities

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 11811