Math Expression Image Retrieval via Attention-Based Framework

Caili Wu, Zhao Zhou, Hao Ye, Jing Yang, Liang He

Abstract: Math expression image retrieval concerns not only visual features but also high-level semantic understanding. Considering math expression image retrieval as traditional content-based image retrieval may suffer the layout misunderstanding, as math expressions with same symbols but different layouts may be interpreted as different meaning. In this paper, we propose a novel retrieval indexing framework for math expression retrieval, namely Scanner-Recognizer-Embedding (SRE) framework. The math expression images passed through SRE are projected into a low dimension semantic space. Retrieval based on embedded semantic vectors is fast and accurate. Experiments on a math expression database demonstrate that the SRE framework outperforms state-of-the-art image-based features.