Cross Attention with Deep Local Features for Few-Shot Image Classification

Tengfei Chu, Hao Shen, Jing Lv, Ming Yang

Abstract: Traditional few-shot learning methods that rely on image-level features have been widely adopted, but they may not be effective in representing the local information of images. Recently, some methods have introduced deep local features that are semantically rich and achieved promising results. However, these methods typically take all local features into consideration, ignoring that some of them, such as sky and grass, are task-irrelevant and may affect the accuracy of image classification. In this thesis, we propose a novel Local Cross Attention Network (LCAN) that aims to learn the query local features that are most relevant to each task. Specifically, we designed a local cross attention mechanism composed of two modules: a query local attention module and a class relevant module. The former is used to determine what kind of query local features to attend by using the spatial and channel information in the query feature, while the latter utilizes the local relationship between the query feature and the support feature to determine which query local features to attend. Extensive experimental on three widely used few-shot classification benchmarks (miniImageNet, tieredImageNet and CUB-200) demonstrate that our proposed method achieves state-of-the-art performance.