Unsupervised Feature Learning for Visual Sign Language Identification

Binyam Gebrekidan Gebre, Onno Crasborn, Peter Wittenburg, Sebastian Drude, Tom Heskes

Abstract: Prior research on language identification focused primarily on text and speech. In this paper, we focus on the visual modality and present a method for identifying sign languages solely from short video samples. The method is trained on unlabelled video data (unsupervised feature learning) and using these features, it is trained to discriminate between six sign languages (supervised learning). We ran experiments on short video samples involving 30 signers (about 6 hours in total). Using leave-one-signer-out cross-validation, our evaluation shows an average best accuracy of 84%. Given that sign languages are underresourced, unsupervised feature learning techniques are the right tools and our results indicate that this is realistic for sign language identification.