EDSL: An Encoder-Decoder Architecture with Symbol-Level Features for Printed Mathematical Expression Recognition

Yingnan Fu, Tingting Liu, Ming Gao, Aoying Zhou

Abstract: Printed mathematical expression recognition (PMER) aims to transcribe a printed mathematical expression image into a structural expression. The task is useful in a wide spectrum of applications, including personalized question recommendation and automatic problem solving. In this paper, we propose a new method named EDSL, shorted for an Encoder-Decoder architecture with Symbol-Level features, to recognize printed mathematical expressions from input images. Its encoder consists of a segmentation module to identify all symbols and their spatial information from the image in an unsupervised manner, and a reconstruction module to recover symbol dependencies after symbol segmentation. Furthermore, we employ a position correction attention mechanism to capture the spatial relationship between symbols, and apply a transformer model to alleviate the negative impact from long output. We conduct extensive experiments on two real datasets to verify the effectiveness and rationality of our proposed EDSL model. The experimental results illustrated that EDSL outperformed state-of-the-art methods by an accuracy margin of $$3.47\%$$ and $$4.04\%$$ in the two datasets, respectively.