Uncertainty-Aware Deep Video Compression with Ensembles

Wufei Ma, Jiahao Li, Bin Li, Yan Lu

Keywords: Video compression, uncertainty, ensemble learning

Abstract: Deep learning-based video compression is a challenging task and many previous state-of-the-art learning-based video codecs use optical flows to exploit the temporal correlation between successive frames and then compress the residual error. Although these two-stage models are end-to-end optimized, errors in the intermediate errors are propagated to later stages and would harm the overall performance. In this work, we investigate the inherent uncertainty in these intermediate predictions and present an ensemble-based video compression model to capture the predictive uncertainty. We also propose an ensemble-aware loss to encourage the diversity between ensemble members and investigate the benefit of incorporating adversarial training in the video compression task. Experimental results on 1080p sequences show that our model can effectively save bits by more than 20% compared to DVC Pro.

One-sentence Summary: We investigate the inherent uncertainty in two-stage video compression models and propose an ensemble-based method to effectively capture the predictive uncertainty and to save bits by an average of 15% compared to DVC Pro.