Facial Action Unit recognition by relation modeling from both qualitative knowledge and quantitative data

Zhen Gao, Shangfei Wang, Chongliang Wu, Jun Wang, Qiang Ji

Abstract: In this paper, we propose to capture Action Unit (AU) relations existing in both qualitative knowledge and quantitative data through Credal Networks (CN). Each node of the CN represents an AU label, and the links and probability intervals capture the probabilistic dependencies among multiple AUs. The structure of CN is designed based on prior knowledge. The parameters of CN are learned from both knowledge and ground-truth AU labels. The AU preliminary estimations are obtained by an existing image-driven recognition method. With the learned credal network, we infer the true AU labels by combining the relationships among labels with the previous obtained estimations. Experimental results on the CK+ database and MMI database demonstrate that with complete AU labels, our CN model is slightly better than the Bayesian Network (BN) model, demonstrating that credal sets learned from data can capture uncertainty more reliably; With incomplete and error-prone AU annotations, our CN model outperforms the BN model, indicating that credal sets can successfully capture qualitative knowledge.