GMiRec: A Multi-image Visual Recommendation Model Based on a Gated Neural Network

Caihong Mu, Xin Tang, Jiashen Luo, Yi Liu

Abstract: Making use of visual perception in recommender systems is becoming more and more important. In the existing visual recommendation models (VRMs), the visual features of items are usually extracted based on the pre-trained convolutional neural network, and then combined with the non-visual features modeling to complete the prediction of users’ interest. There are two challenges in this field so far. First, most VRMs are developed around single-image items, and how to more effectively mine the visual features of multi-image items is seldom considered. Second, most models do not consider the distribution difference between the training datasets of the pre-trained model and the datasets for recommendation when extracting visual features based on the pre-training model, which may deepen the gap in the convolutional neural network’s understanding of image semantics on datasets. To address the above challenges, a Multi-image Visual Recommendation Model based on a Gated Neural Network (GMiRec) is proposed. It performs different forms of pooling operations on the visual features of multi-image items and uses the feed-forward neural network to realize the fusion of the multi-image visual information. In addition, a gated neural network taking item categories as input is designed to achieve supervised dimensionality reduction on the item visual features, which alleviates the problem of semantic gap. Experiments conducted on the Amazon datasets show that the proposed model is significantly improved compared with the existing models.