TST: Tree Structured Transformer for Handwritten Mathematical Expression Recognition

Yejing Xie, Harold Mouchère

Abstract: Handwritten Mathematical Expression Recognition (HMER) is a challenging task due to the complexity of mathematical symbols and the variability of handwriting styles. Tree Structured Transformer (TST) is proposed to recognize handwritten mathematical expressions by predicting the Symbol Layout Tree (SLT) of the expression. The proposed method is based on an Encoder-Decoder architecture, where the Decoder is designed to predict the tree structure of the expression, which predicts the node labels, edge labels, and parent index of each node. This TST Decoder can be combined with different Encoder to handle both online or offline handwritten mathematical expressions with CNN-based Encoder for offline HMER and Graph-based Encoder for online HMER. Both the online and offline systems are evaluated on the CROHME 2023 dataset, and evaluated by the standard evaluation tools, achieving competitive results compared to the state-of-the-art methods.

External IDs:doi:10.1007/978-3-032-04630-7_14