Go to ACL ARR 2025 February homepageA Novel Matching Paradigm: Unified Generative and Discriminative Large Language Model with Plug-and-Play Fine-Tuning
Matching paradigm plays a crucial role in large-scale information retrieval and has been widely deployed in industrial search engines. Limited by the feature interaction ability and discriminative architecture, the traditional two-tower and single-tower matching paradigms can no longer satisfy the demands for the performance and interpretability of matching paradigms in the era of LLMs. Existing approaches attempt to utilize LLMs merely as feature extractors, which falls short of fully leveraging the capabilities of LLMs. Therefore, we propose a novel matching paradigm: unified generative and discriminative large language model with plug-and-play fine-tuning (UGD). It integrates the two-tower, single-tower and generative tasks within the same LLM framework through the attention map partition, so as to achieve the deep traction of generative tasks to discriminative tasks and the distillation of single-tower to two-tower discrimination by the plug-and-play multi-task fine-tuning mechanism. To support the training of UGD, we also reconstruct six text matching datasets by appending reason labels based on ERNIE-4.0-Turbo-8K. Extensive experimental results demonstrate that UGD has far superior performance and comparable interpretability. And it has been applied to the industrial search engine, leading to a remarkable enhancement of search experience. Open access upon publication.