Automatic Image Recommendation for Economic Topics using Visual and Semantic Information

Chan Hur, Changhun Hyun, Hyeyoung Park

Abstract: This paper proposes an image recommendation system for automatic report generation on economic topics. For a given specific headline query on daily economic events, the proposed system collects candidate images through a public search engine and choose the most appropriate one for summary report on the event. The proposed system is composed of two deep learning-based modules of different modalities: image filtering module and text matching module. In the image filtering module, an image classifier network is adopted to filter out non-photo images such as graph and tables. In the text matching module, a sentence embedding network is adopted to get text query vector and image caption vector and calculate their matching scores. By analyzing image and text information together, the proposed system can recommend suitable images both visually and semantically. Through computational experiments using a number of recent economic topics, we confirm that recommended images of the proposed system are more appropriate than that of conventional search engine.