Fusing Visual Saliency for Material Recognition

Lin Qi, Ying Xu, Xiaowei Shang, Junyu Dong

Abstract: Material recognition is researched in both computer vision and vision science fields. In this paper, we investigated how humans observe material images and found the eye fixation information improves the performance of material image classification models. We first collected eye-tracking data from human observers and used it to fine-tune a generative adversarial network for saliency prediction (SalGAN). We then fused the predicted saliency map with material images, and fed them to CNN models for material classification. The experiment results show that the classification accuracy is improved than those using original images. This indicates that human's visual cues could benefit computational models as priors.