Self-supervised Label-Visual Correlation Hashing for Multi-label Image Retrieval

Yu Liu, Yanzhao Xie, Jingkuan Song, Rukai Wei, Ke Zhou

Abstract: Perceiving multiple objects within an image without the labels’ supervision is the challenge of multi-label image hashing tasks. Existing unsupervised hashing approaches do reconstruction or contrastive learning for the representation of the object of interest but ignore the other objects in the image. We propose to use pseudo labels to provide candidate objects, making the image match the possible objects’ features by the co-occurrence correlations between labels. As a result, we explore the co-occurrence correlations based on empirical models and design a data augmentation strategy in a self-supervised learning framework to learn label-level embeddings. We also build the image visual correlations and design a dual overlapping group sum-pooling (OGSP) component to fuse label-level and visual-level embeddings into image representations, alleviating noise from empirical models. Extensive experiments on public multi-label image datasets using pseudo labels demonstrate that our self-supervised label-visual correlation hashing framework outperforms state-of-the-art label-free hashing algorithms for retrieval. GitHub address: https://github.com/lzHZWZ/SS-LVH.git.