A Novel Patch Selection Approach for Quality Assessment on Short-Form Videos

Shijie Wen, Lai Jiang, Minglang Qiao, Mai Xu, Xin Deng, Shengxi Li, Yiping Duan

Abstract: Video quality assessment (VQA) on short-form videos plays a critical role in optimizing online multimedia systems. Traditional VQA approaches have made significant progress with the widespread applications of deep learning. However, the quality assessment of short-form videos has re-ceived insufficient research attention with limited performance, neglecting the unique perceptual characteristics of human over short-form videos, e.g., large transfer distance and scattered distribution of fixations. In this paper, we propose the patch selection VQA (PS-VQA) approach to promote this field by incorporating human attention mechanisms during short-form video viewing. In our PS-VQA approach, we first introduce a patch selection network (PS-Net) to identify the patches most relevant to perceived video quality in short-form videos. Subsequently, we develop a quality assessment network (QA-Net) to extract both local and global quality-aware features from the selected patches to predict the overall video quality. Extensive experimental results validate the effectiveness of the proposed VQA approach on short-form videos. Interestingly, our ablation experiments show that accurately assessing the quality of a short-form video merely requires 5 % pixels of the whole frame.