AUPro: Multi-label Facial Action Unit Proposal Generation for Sequence-Level Analysis

Yingjie Chen, Jiarui Zhang, Diqi Chen, Tao Wang, Yizhou Wang, Yun Liang

Abstract: Facial action unit (AU) plays an essential role in human facial behavior analysis. Despite the progress made in frame-level AU analysis, the discrete classification results provided by previous work are not explicit enough for the analysis required by many real-world applications, and as AU is a dynamic process, sequence-level analysis maintaining a global view has yet been gravely ignored in the literature. To fill in the blank, we propose a multi-label AU proposal generation task for sequence-level facial action analysis. To tackle the task, we design AUPro, which takes a video clip as input and directly generates proposals for each AU category. Extensive experiments conducted on two commonly used AU benchmark datasets, BP4D and DISFA, show the superiority of our proposed method.