Quantizable deep representation learning with gradient snapping layer for large scale search

Shicong Liu, Hongtao Lu

Abstract: Recent advance of large scale similarity search requires to learn deep representations that both strongly preserve similarities between data pairs and can be accurately quantized via vector quantization. Existing methods simply leverage quantization loss and similarity loss, which result in unexpectedly biased back-propagating gradients and affect the search performances. To this end, we propose a novel gradient snapping layer (GSL) to regularize the back-propagating gradient towards a neighboring codeword, the generated gradients works better on reducing similarity loss and also propel the learned representations to be accurately quantized. Joint deep representation and vector quantization learning can be easily performed by alternatively optimizing the quantization codebook and the deep neural network. The proposed framework is compatible with various existing vector quantization approaches. Experimental results on various standard benchmark datasets demonstrate that the proposed framework is effective, flexible and outperforms the state-of-the-art large scale similarity search methods.