Training for Compositional Sensitivity Reduces Dense Retrieval Generalization

Radoslav Ralev, Aditeya Baral, Iliya Sotirov Zhechev, Jen Agarwal, Srijith Rajamohan

Keywords: natural language processing, representation learning, limitations, empirical study

TL;DR: We validate empirically previously established theoretical limitations of representation learning in the context of natural language processing.

Abstract: Dense retrieval compresses texts into single embeddings ranked by cosine similarity. While efficient for recall, this interface is brittle for identity-level matching: minimal compositional edits (negation, role swaps) flip meaning yet retain high similarity. Motivated by geometric results for unit-sphere cosine spaces, we test this retrieval-composition tension in text-only retrieval. Across four dual-encoder backbones, adding structure-targeted negatives consistently reduces zero-shot NanoBEIR retrieval (8--9\% mean nDCG@10 drop on small backbones; up to 40\% on medium ones), while only partially improving pooled-space separation. Treating pooled cosine as a recall interface, we then benchmark verifiers scoring token--token cosine maps. MaxSim (late interaction) excels at reranking but fails to reject structural near-misses, whereas a small Transformer over similarity maps reliably separates near-misses under end-to-end training.

Anonymization: This submission has been anonymized for double-blind review via the removal of identifying information such as names, affiliations, and identifying URLs.

Style Files: I have used the style files.

Submission Number: 59