Deep Pixel Restoration Loop Coding Network [Application Notes] [Application Notes]

Tianxiang Wang, Qun Dai, Muxuan Yuan

Abstract: This paper presents a solution to the video prediction problem based on the deep learning paradigm. Predicted frames generated by existing video prediction models are often blurry and have difficulty maintaining accuracy in multi-step prediction. To overcome these limitations, this paper presents a deep learning model, named the deep pixel restoration loop coding network (DPR-LC-Net), which employs the concept of predictive coding and adopts pixels in the real frames. While making a long-term prediction, it can generate clear prediction frames with few errors. The calculation process of DPR-LC-Net is multi-sequential: it completes calculations in the form of an approximate loop from top to bottom and then from left to right. After predicting subsequent steps, DPR-LC-Net both calculates errors and removes them from sequential prediction. Finally, the model includes a unique pixel restoration module that works efficiently on pixels in the preceding real frames to generate predicted frames, thereby improving the clarity of the predicted frames. Extensive experiments using four video datasets illustrate that the prediction performance of DPR-LC-Net is superior to that of the state-of-the-art models.