Learning Neural Networks for Multi-label Medical Image Retrieval Using Hamming Distance Fabricated with Jaccard Similarity Coefficient

Asim Manna, Debdoot Sheet

Abstract: Deep neural hashing (DNH) has demonstrated its effectiveness in content-based medical image retrieval (CBMIR) for efficient nearest-neighbor search in large image datasets. It learns a hash function to generate hash codes from the images. Conventional pairwise DNH methods are inadequate for multi-label CBMIR as they do not incorporate between the Hamming distance (HD) of hash codes and the Jaccard similarity coefficient (JSC) of label sets for an image pair. This work introduces a JSC-based loss function called adaptive HD loss (AHDL) for learning HD between hash pairs using a deep neural network to retrieve multi-label medical images. AHDL helps the model assign an appropriate HD between a pair of hash codes based on their image similarity level. We also adopt pairwise multi-label classification loss to generate unique features for each class combination. Experiments are demonstrated on the publicly available NIH chest X-ray dataset. Our method achieves \(3.98\%\) higher normalized discounted cumulative gain compared to the state-of-the-art method for a top-100 image retrieval task.