Towards Efficient Fairness Image Retrieval with Disentangled Information Suppression

ICLR 2026 Conference Submission17422 Authors

Keywords: Fairness, Disentangled Representation, Image Retrieval

TL;DR: DISH is a deep hashing framework that improves both retrieval accuracy and group fairness by learning disentangled, information-suppressed representations prior to binarization.

Abstract:

Deep hashing has emerged as an effective method for large-scale image retrieval, improving computational efficiency by converting high-dimensional data into compact binary codes. Despite its success, recent studies reveal that deep hashing methods may exhibit fairness issues, leading to biased or discriminatory retrieval results across demographic groups. To jointly improve retrieval accuracy and group fairness, we introduce Disentangled Information Suppressed Hashing (DISH), a framework that learns fair and discriminative representations. DISH employs a disentangled encoder to decompose each image into factor-specific representations. To encourage semantic concentration and interpretability, a disentangled consistency objective is introduced to enforce factor-level stability under augmentation and align semantic evidence with latent factors.
Furthermore, an information suppression module is designed to mitigate sensitive information leakage through probability-driven channel masking, channel-wise adversarial learning, and conditional covariance regularization. These components work collaboratively to eliminate sensitive signals both within and between feature channels while preserving semantic discriminability. Extensive experiments on multiple benchmarks show that DISH substantially outperforms state-of-the-art deep hashing baselines in retrieval accuracy while achieving better fairness.

Primary Area: alignment, fairness, safety, privacy, and societal considerations

Submission Number: 17422