On Unique Decodability

Marco Dalai, Riccardo Leonardi

Abstract: In this paper, we propose a revisitation of the topic of unique decodability and of some fundamental theorems of lossless coding. It is widely believed that, for any discrete source X , every ldquouniquely decodablerdquo block code satisfies E [ l ( X 1 , X 2 ,..., X n )]ges H ( X 1 , X 2 ,..., X n ) where X 1 , X 2 ,..., X n are the first n symbols of the source, E [ l ( X 1 , X 2 ,..., X n )] is the expected length of the code for those symbols, and H ( X 1 , X 2 ,..., X n ) is their joint entropy. We show that, for certain sources with memory, the above inequality only holds when a limiting definition of ldquouniquely decodable coderdquo is considered. In particular, the above inequality is usually assumed to hold for any ldquopractical coderdquo due to a debatable application of McMillan's theorem to sources with memory. We thus propose a clarification of the topic, also providing an extended version of McMillan's theorem to be used for Markovian sources.