Semantic Similarity-based Visual Reasoning without Language Information

ChangSu Choi, HyeonSeok Lim, Hayoung Jang, Juhan Park, Eunkyung Kim, Kyungtae Lim

Abstract: In this research, we propose new training data for the visual reasoning task based on semantic similarity and proposed a deep learning model that utilizes the data. The first contribution of this study is the construction of training data. Based on a total of 40 object attributes, we created a visual inference problem using only image data. As a result, a total of 6,000 datasets were built to create training and test data. We also propose a visual inference model as the second contribution of this work. The inference model shown in this study was evaluated for two tasks using ResNet50 and Vision Transformer, respectively. Based on the experimental evaluation results, we investigated the suitable pre-trained model for both single-choice binary reasoning and multiple-selection reasoning, respectively.

External IDs:dblp:conf/icaiic/ChoiLJPKL23