EffiSign network: a comprehensive approach for sign language recognition

Bhumika Karsh, Rabul Hussain Laskar, Ram Kumar Karsh, Manas Kamal Bhuyan

Abstract: Sign language recognition (SLR) is crucial for connecting the deaf and hearing communities. Achieving efficient hand gesture recognition is challenging due to variations in lighting, backgrounds, hand sizes, shapes, and similarities among gestures. To address these challenges, a two-stage framework is proposed. In the first stage, a modified automatic GrabCut is employed to segment the hand region from complex backgrounds, thereby removing unwanted noise. The second stage performs accurate hand gesture classification. An enhanced model named EffiSign, based on EfficientNet-B7, is introduced. This architecture leverages a compound scaling technique to simultaneously optimize depth, width, and resolution. Performance is further improved through fine-tuning by selectively unfreezing layers from specific blocks. EffiSign was evaluated on multiple datasets, including MUGD, NUS-II, ISL, and ArASL, achieving mean accuracies of 97.22%, 100%, 99.59%, and 97.75%, respectively, with improvements of 4.22%, 1.02%, 2.83%, and 1.92% over prior methods. Comparative analysis demonstrates that EffiSign surpasses recent state-of-the-art models in accuracy and robustness. Additionally, a graphical user interface (GUI) application was developed to translate sign language gestures into English text, showcasing the applicability of the proposed approach.

External IDs:dblp:journals/mta/KarshLKB26