Learning to rank under uncertainty: a robust neural approach

Maryam Khodabakhsh, Ebrahim Bagheri

Abstract: Dense retrieval models provide representations in the form of embeddings in latent space and output a single deterministic score for a document based on the estimation of its relevance to the input query. While remarkable progress has been achieved in dense retrieval methods, they are limited by the fact that they consider queries and documents as deterministic points in latent space that encode the most likely features of the given query or document, and hence do not explicitly encode any degrees of noise, ambiguity or uncertainty. In this paper, we build on existing strong transformer-based dense retrievers by enabling them to capture uncertainty in latent space. In our proposed approach, embeddings in latent space are no longer a deterministic point, but rather a probabilistic distribution. With such probabilistic embeddings, the dense retrievers can be trained to achieve competitive performance on in-distribution queries and higher generalizability on out-of-distribution queries. Based on extensive experiments, we demonstrate that our proposed model consistently improves retrieval effectiveness in comparison to the state-of-the-art dense retrieval methods.

External IDs:dblp:journals/kais/KhodabakhshB25