Facial expression recognition with local prominent directional pattern

Farkhod Makhmudkhujaev, Mohammad Abdullah-Al-Wadud, Md. Tauhid Bin Iqbal, Byungyong Ryu, Oksam Chae

Abstract: Highlights • A novel feature extraction method using a local histogram of directional variations is proposed. • The feature encoding process is discriminative, and stable against local distortions. • An adaptive thresholding process for avoiding feature extraction at featureless flat/smooth regions is introduced. • A feature selection mechanism that retains expression-related micro-level features is proposed. • Extensive experiments on various datasets including posed, spontaneous, noisy and position-varied facial expression images are conducted. Abstract Local edge-based descriptors have gained much attention as feature extraction methods for facial expression recognition. However, such descriptors are found to suffer from unstable shape representations for different local structures for their sensitivity to local distortions such as noise and positional variations. We propose a novel edge-based descriptor, named Local Prominent Directional Pattern (LPDP), which considers statistical information of a pixel neighborhood to encode more meaningful and reliable information than the existing descriptors for feature extraction. More specifically, LPDP examines a local neighborhood of a pixel to retrieve significant edges corresponding to the local shape and thereby ensures encoding edge information in spite of some positional variations and avoiding noisy edges. Thus LPDP can represent important textured regions much effectively to be used in facial expression recognition. Extensive experiments on facial expression recognition on well-known datasets also demonstrate the better capability of LPDP than other existing descriptors in terms of robustness in extracting various local structures originated by facial expression changes.