Quality-aware pattern diffusion for video object segmentation

Chuanwei Zhou, Chunyan Xu, Jun Li, Zhen Cui, Jian Yang

Abstract: Recently, great progresses have been achieved under the support of memory mechanism when dealing with video object segmentation (VOS) problem. Despite its achievements, existing VOS approaches still suffer from abnormal samples, which derive from intrinsic video artifacts such as occlusion and motion blur. To mitigate the above issue, in this paper, we propose a quality-aware pattern diffusion (QPD) framework to boost the VOS performance. To achieve quality-aware pattern diffusion, a quality alignment mechanism is proposed, and it aims to promote the contributions of those normal samples while suppressing those abnormal ones during the feature propagation/diffusion processes. With our proposed quality alignment mechanism, the diffused instance features could be kept staying in the normal feature space, keeping from feature contamination caused by those low-quality samples. We first introduce a learnable quality evaluator to assess the sample qualities in both the temporal domain (i.e., across the historical frames), as well as the spatial domain (i.e., within the current frame). To achieve adaptive historical feature propagation into the current instance, a quality-aware long-term context propagation module is proposed, with which more stable instance representations could be achieved through the established quality-aware feature propagation process. A quality-aware pattern diffusion module is further introduced to address the spatial-domain abnormal samples, resulting in effective decoder feature refinement through building the quality-aware correspondence weights. Extensive experiments have demonstrated that our proposed quality alignment mechanism could boost the performance by a great margin over a strong baseline while achieving state-of-the-art performances on public VOS benchmarks.