Improving Dual-Encoder Training through Dynamic Indexes for Negative Mining

Nicholas Monath, Manzil Zaheer, Kelsey Allen, Andrew McCallum

Abstract: Dual encoder models are ubiquitous in modern classification and retrieval. Crucial for training such dual encoders is an accurate estimation of gradients from the partition function of the softmax ...