Accelerating Video Object Segmentation with Compressed Video

Kai Xu, Angela Yao

Abstract: We propose an efficient plug-and-play acceleration framework for semi-supervised video object segmentation by exploiting the temporal redundancies in videos presented by the compressed bitstream. Specifically, we propose a motion vector-based warping method for propagating seg- mentation masks from keyframes to other frames in a bi- directional and multi-hop manner. Additionally, we in- troduce a residual-based correction module that can fix wrongly propagated segmentation masks from noisy or er- roneous motion vectors. Our approach is flexible and can be added on top of several existing video object segmenta- tion algorithms. We achieved highly competitive results on DAVIS17 and YouTube-VOS on various base models with substantial speed-ups of up to 3.5X with minor drops in ac- curacy.