Hyp-DAN: Hyperbolic Distance-Aware Attention Networks

Fuchuan Xiang, Jianhang Tang, Shaobo Li, Guoyin Wang, Ji Xu

Abstract: Hyperbolic embedding has advantages for hierarchically category images, hence it has been applied in few-shot learning and achieved significant results. However, treating all samples equally may not ensure that the learned hyperbolic embedding adequately considers various modalities within the same category. To address this issue, this paper proposes a novel hyperbolic attention mechanism for few-shot learning, which adjusts weights based on the hyperbolic distance of samples to the average position. This approach balances the typicality and diversity of labeled samples, aiding the model in a deeper understanding of data structures. After hyperbolic embedding, weights were redistributed through this attention mechanism for few-shot learning. Experiments were conducted on the CUB and miniImageNet datasets. The experiments demonstrate the superiority of the proposed attention mechanism when using hyperbolic embedding for hierarchically category data.