Retrieval-Enhanced Machine Learning: Synthesis and Opportunities

Fernando Diaz, Andrew Drozdov, To Eun Kim, Alireza Salemi, Hamed Zamani

Abstract: Retrieval-enhanced machine learning (REML) refers to the use of information retrieval methods to support reasoning and inference in machine learning tasks. Although relatively recent, these approaches can substantially improve model performance. This includes improved generalization, knowledge grounding, scalability, freshness, attribution, interpretability and on-device learning. To date, despite being influenced by work in the information retrieval community, REML research has predominantly been presented in natural language processing (NLP) conferences. Our tutorial addresses this disconnect by introducing core REML concepts and synthesizing the literature from various domains in machine learning (ML), including, but beyond NLP. What is unique to our approach is that we used consistent notations, to provide researchers with a unified and expandable framework. The tutorial will be presented in lecture format based on an existing manuscript, with supporting materials and a comprehensive reading list available at https://retrieval-enhanced-ml.github.io/SIGIR-AP2024-tutorial.