A Parallelization Framework for High Throughput Entropy Coding

Amir Said, Abo Talib Mahfoodh

Abstract: We propose a general framework for parallel entropy coding in media compression, which preserves compression efficiency, and is well matched to future generations of general-purpose or custom processors. Similarly to some previous parallelization methods, it is based on the fact that optimal compression is not affected by the arrangement of coded bits, but it goes further in exploiting the decreasing cost of data processing and memory. We use finite-state-machine models for identifying the best manner of separating data into segments that can be processed independently, while minimizing compression losses. Additional advantages include the ability to use, within this framework, increasingly more complex data modeling techniques, and the freedom to mix different types of coding. We confirm the parallelization effectiveness using coding simulations that run on multi-core processors, and show how throughput scales with the number of cores.