Improving Online Handwriting Recognition with Transfer Learning Using Out-of-Domain and Different-Dimensional Sources

Jiseok Lee, Masaki Akiba, Brian Kenji Iwana

Abstract: Online handwriting recognition is a widely used technique in our daily lives. Furthermore, deep learning has become one of the most popular and influential methods for online handwriting recognition. However, artificial neural networks typically require massive datasets. Transfer learning is a standard method to overcome the problem of lack of data. Usually, transfer learning works by initiating a network with trained weights and fine-tuning with a smaller dataset. Still, obtaining large amounts of online handwriting can be difficult for pre-training networks. Therefore, we propose pre-training with data sources with dimensions different from handwriting. Namely, we propose using univariate or multivariate data as a source dataset for two-dimensional target data by embedding out-of-domain time series of different dimensions into two-dimensional space. We evaluated the proposed method with four handwritten character datasets: a numerical digit dataset, an uppercase alphabet dataset, a lowercase alphabet dataset, and a Chinese character dataset. Through the evaluation, we demonstrate that transfer learning from datasets with a different dimensionality as online handwriting is possible.