Long-Tailed Recognition Based on Self-attention Mechanism

Zekai Feng, Hong Jia, Mengke Li

Abstract: The long-tailed distribution data poses significant challenges for visual classification tasks. The existing solutions can be categorized into three main categories, i.e., class re-balancing, information augmentation, and module improvement. Recent breakthroughs have been made in applying decoupled structures to long-tailed recognition. Inspired by decoupled structures’ performance and attention mechanisms’ remarkable performance in visual tasks, we propose a novel module improvement approach, a self-attention-based long-tailed recognition network. In this work, we extract feature information from the tail class samples and incorporate this information into the deep network. The findings are surprising: Adding a self-attention layer to an existing deep network makes it possible to achieve a more robust ability for long-tailed recognition. Although the method we use to extract features from tail classes is simple, the feature information obtained from these classes proves highly effective in long-tailed recognition tasks. We conduct extensive experiments, systematically exploring how attention mechanisms influence long-tailed recognition. We also analyze the similarities and differences between our proposed method and other current attention mechanisms.