Indian Regional Sign Language Recognition

Sujatha C, Madhura K. Patil, Umadevi F. M, Neha Jain, Akshata Badni, Ranjeeta Angadi, Shankar Gangisetty

Abstract: Sign Language is a medium of communication in the Deaf and Hard of Hearing community (DHH community). According to WHO, there are approximately 63 million people in India, including 3.3 million from the state of Karnataka, with 0.3 million children belonging to DHH community. The sign language gestures vary globally, making communication difficult between the DHH community and others. Hence, Sign Language Recognition has gained significant attention in recent years. There is a limited publicly available dataset on Indian Sign Language. In addition, the dataset for regional languages especially the Kannada dataset, is unavailable for the sign recognition tasks. Hence initially, in our work, we built a dataset of Indian Regional Kannada Sign Language (IRKSL) for different levels of data at character, word, and sentence levels. We propose a method for recognizing the Indian sign language and translating it into Kannada regional language. We demonstrate the proposed method on the IRKSL dataset and obtained an accuracy of 81.69% and 73.87% with and without feature augmentation, respectively, on the IRKSL dataset. We also evaluated our work on the INCLUDE 50 dataset and obtained a better accuracy of 93.42% as compared to the state-of-art method.