Learning Symbol Relation Tree for Online Handwritten Mathematical Expression Recognition

Thanh-Nghia Truong, Hung Tuan Nguyen, Cuong Tuan Nguyen, Masaki Nakagawa

Abstract: This paper proposes a method for recognizing online handwritten mathematical expressions (OnHME) by building a symbol relation tree (SRT) directly from a sequence of strokes. The recognition system has two parts: a temporal classifier and a tree connector. The temporal classifier uses global context to produce a sequence of symbols and spatial relations between symbols from an OnHME pattern. It is a bidirectional recurrent neural network trained from multiple derived paths of SRTs. The tree connector splits the sequence into several sub-SRTs and connects them to form the SRT by looking up the best combination among those sub-SRTs. Besides, we adopt a tree sorting method to deal with various stroke orders. Recognition experiments indicate that the proposed OnHME recognition system is competitive to other methods. The recognition system achieves 44.12% and 41.76% expression recognition rates on the Competition on Recognition of Online Handwritten Mathematical Expressions (CROHME) 2014 and 2016 testing sets, respectively.