An Optimization Approach for Elementary School Handwritten Mathematical Expression Recognition

Sérgio F. Chevtchenko, Ruan Carvalho, Luiz Rodrigues, Everton Souza, Daniel Carneiro Rosa, Filipe R. Cordeiro, Cicero Pereira, Thales Vieira, Marcelo Marinho, Diego Dermeval, Ig Ibert Bittencourt, Seiji Isotani, Valmir Macario

Abstract: This study introduces a novel approach to Handwritten Mathematical Expression Recognition (HMER), focusing on elementary school mathematical expressions. Recognizing the challenges posed by limited training data and the unique characteristics of elementary students’ handwriting, we present a multiobjective optimization method tailored for small training datasets. We employ state-of-the-art HMER methods, including transformer-based and attention mechanism models, and optimize them using a custom dataset comprised of elementary school arithmetic equations. This dataset contains 1237 images and includes both horizontal and vertical equations and isolated numbers, featuring common errors in children’s handwriting. Additional similar datasets are also leveraged for training augmentation. Our experimental results demonstrate the efficacy of the optimization approach, significantly improving the performance of the evaluated models in terms of expression recognition rate and inference speed. This study contributes to the field of HMER by providing an effective optimization approach for SOTA models and by introducing a specialized dataset for elementary school mathematics. The dataset is available upon request.