Computational face reader based on facial attribute estimation

Xiangbo Shu, Yunfei Cai, Liu Yang, Liyan Zhang, Jinhui Tang

Abstract: Chinese face reading has demonstrated the often satisfying capabilities to tell the characteristics (mostly exaggerated as fortune) of a person by reading his/her face, i.e. understanding the fine-grained facial attributes (e.g., length of nose, single/double-fold eyelid, density of eyebrows, etc.). Thus, a smart face reading system should estimate the fine-grained facial attributes well. Therefore, In this paper, we first study the fine-grained facial attribute estimation problem and propose a novel deep convolutional network equipped with a new facial region pooling layer (called FRP-net), to accurately estimate the fine-grained facial attributes. To capture the characteristics of fine-grained facial attributes, the embedded FRP layer implements the pooling operation on the searched facial region windows (locates the region of each facial attribute) instead of the commonly-used sliding windows. Further, we push the proposed fine-grained facial attribute estimation method into the face reading problem and present a computational face reader system to automatically infer the characteristics of a person based on his/her face. For example, it can estimate the attractive and easy-going characteristics of a Chinese person from his/her big eyes according to the Chinese anthroposcopy literature. The experimental results on facial attribute estimation demonstrate the superiority of the proposed FRP-net compared to the baselines, and the qualitative and quantitative evaluations on face reading validate the excellence of the presented face reader system.