Efficient prediction algorithm of integer DCT coefficients for H.264/AVC optimization

Hanli Wang, Sam Kwong, Chi-Wah Kok

Abstract: This paper presents a novel efficient prediction algorithm to reduce redundant discrete cosine transform (DCT) and quantization computations for H.264 encoding optimization. A theoretical analysis is performed to study the sufficient condition for DCT coefficients to be quantized to zeros. As a result, three sufficient conditions corresponding to three types of transform and quantization methods in H.264 are proposed. Compared with other algorithms in the literature, the proposed algorithm derives more precise and efficient conditions to predict zero quantized DCT coefficients. Both the theoretical analysis and experimental results demonstrate that the proposed algorithm is superior to other algorithms in terms of the computational complexity reduction, encoded video quality, false acceptance rate, and false rejection rate.