A Lightweight Model for Deep Frame Prediction in Video Coding

Hyomin Choi, Ivan V. Bajic

Abstract: Recent studies have demonstrated the efficacy of deep neural network (DNN)-based inter frame prediction for video coding. The network commonly used in these studies is built upon a U-Net-like architecture and produces content-adaptive 1-D separable filters with a large number of taps for frame prediction. This leads to a model with a large number of parameters. In this paper, we propose a lighter version of the network with significantly fewer parameters, by making use of dilated convolutional layers and making the U-Net shallower. In addition, we introduce a DCT-based ℓ1-loss term that encourages compression, and explore several ways of integrating our lightweight model into HEVC. Both frame prediction accuracy and coding efficiency are compared against previous works. The experiments show that the proposed model achieves up to 6.4% average bit reduction in terms of BD-Bitrate against HEVC, which is significantly better than existing methods in the literature.