Sig-RFPNet: multi-branch network and multi-classifier method for offline signature recognition

Wanying Li, Mahpirat Muhammat, Xuebin Xu, Alimjan Aysa, Kurban Ubul

Abstract: Due to the problems of large intra-class variations, small inter-class variations, sparse effective features, and small training samples, handwritten signatures are difficult to further improve recognition accuracy. We attempt to compensate for the impact of the above factors and improve recognition accuracy by using a multi-branch network structure and multi-classifier algorithm. This method increases the diversity of features at different scales and utilizes the common decision-making mechanism of multiple classifiers. Firstly, a multi-branch network structure based on residual network and feature pyramid network is proposed. After extracting deep features, a global average pooling layer is used instead of fully connected layer for classification, reducing model parameters and improving generalization performance. Secondly, different weights are applied to the losses of each classifier based on the scale differences of features extracted from different branches. Multiple classifiers are used to predict the top-K class probabilities, and the class with the largest probability sum is taken as the recognition result to further improve accuracy. Experiments are conducted on publicly available Latin datasets, CEDAR and MCYT, by dividing the training and testing sets according to the percentage ratio and time acquisition order. The proposed method achieved recognition accuracy of over 98.50% on two datasets using only 25% of the training data, and achieved accuracy of over 80.00% using only 2 training samples per writer. These results demonstrate the significant advantages of the proposed method in scenarios with small training samples.