DeSIRe: Deep Signer-Invariant Representations for Sign Language Recognition

Pedro M. Ferreira, Diogo Pernes, Ana Rebelo, Jaime S Cardoso

Abstract: As a key technology to help bridging the gap between deaf and hearing people, Sign Language Recognition (SLR) has become one of the most active research topics in the human-computer interaction field. Although several SLR methodologies have been proposed, the development of a real-world SLR system is still a very challenging task. One of the main challenges is related to the large inter-signer variability that exists in the manual signing process of sign languages. To address this problem, we propose a novel end-to-end deep neural network that explicitly models highly discriminative signer-independent latent representations from the input data. The key idea of our model is to learn a distribution over latent representations, conditionally independent of signer identity. Accordingly, the learned latent representations will preserve as much information as possible about the signs, and discard signer-specific traits that are irrelevant for recognition. By imposing such regularization in the representation space, the result is a truly signer-independent model which is robust to different and new test signers. Experimental results demonstrate the effectiveness of the proposed model in several SLR databases.